Devices, methods, and graphical user interfaces for accessibility using a touch-sensitive surface

ABSTRACT

A method is performed by an accessible electronic device with a display and a touch-sensitive surface. The method includes: displaying a plurality of user interface elements on the display, wherein a current focus is on a first user interface element; detecting a first finger gesture on the touch-sensitive surface, wherein the first finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a second user interface element; and, in response to detecting the first finger gesture: changing the current focus from the first user interface element in the plurality of user interface elements to the second user interface element in the plurality of user interface elements; and outputting accessibility information associated with the second user interface element.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/184,825, “Devices, Methods, and Graphical User Interfaces forAccessibility Using a Touch-Sensitive Surface,” filed Jun. 7, 2009,which is incorporated by reference herein in its entirety.

This application is related to the following applications: (1) U.S.patent application Ser. No. 12/565,745, “Devices, Methods, and GraphicalUser Interfaces for Accessibility Using a Touch-Sensitive Surface,”filed Sep. 23, 2009, and (2) U.S. patent application Ser. No.12/565,746, “Devices, Methods, and Graphical User Interfaces forAccessibility Using a Touch-Sensitive Surface,” filed Sep. 23, 2009,which are incorporated by reference herein in their entirety.

TECHNICAL FIELD

The disclosed embodiments relate generally to electronic devices forpeople with impaired vision, and more particularly, to electronicdevices that provide accessibility using a touch-sensitive surface, suchas a touch screen display or a track pad.

BACKGROUND

The use of touch-sensitive surfaces as input devices for computers andother electronic devices has increased significantly in recent years. Asthe use of these touch-sensitive surfaces has increased, the need fortouch-based navigation methods that provide accessible navigationfeedback has also increased (e.g., audible feedback, haptic feedback,and/or Braille output), especially for people with impaired vision. Forexample, low-vision users, blind users, dyslexic users or others withlearning disabilities, or even sighted users who simply want or need touse a device without looking at the device during operation can benefitfrom accessibility user interfaces that permit touch-based navigationamong user interface elements on the device. For blind users inparticular, an effective accessibility interface is not merely anoption: it is required to use these devices without assistance fromsighted users.

Unfortunately, existing accessibility interfaces on devices withtouch-sensitive surfaces remain cumbersome and inefficient. Navigationand activation of user interface elements is often difficult, therebycreating a significant cognitive burden on a user with impaired vision.In addition, existing accessibility methods take longer than necessary,thereby wasting energy. This latter consideration is particularlyimportant in battery-operated devices.

Accordingly, there is a need for electronic devices with touch-sensitivesurfaces (e.g., touch screen displays and/or track pads) with faster,more efficient touch-based accessibility methods and interfaces. Suchmethods and interfaces may complement or replace existing accessibilitymethods for navigating and interacting with user interface objects. Suchmethods and interfaces reduce the cognitive burden on a user withimpaired vision and produce a more efficient human-machine interface.For battery-operated devices, such methods and interfaces conserve powerand increase the time between battery charges.

SUMMARY

The above deficiencies and other user interface accessibility problemsassociated with devices with touch-sensitive surfaces are reduced oreliminated by the disclosed devices. In some embodiments, the device isa desktop computer. In some embodiments, the device is portable (e.g., anotebook computer or handheld device). In some embodiments, the devicehas a touchpad (also known as a “track pad”). In some embodiments, thedevice has a touch-sensitive display (also known as a “touch screen” or“touch screen display”). In some embodiments, the device has a graphicaluser interface (GUI), one or more processors, memory and one or moremodules, programs or sets of instructions stored in the memory forperforming multiple functions. In some embodiments, the user interactswith the GUI primarily through finger contacts and gestures on thetouch-sensitive surface. In some embodiments, the functions may includepresenting, word processing, website creating, disk authoring,spreadsheet making, game playing, telephoning, video conferencing,e-mailing, instant messaging, voice memo, web browsing, digital musicplaying, purchasing and/or downloading digital content, and/or digitalvideo playing. Executable instructions for performing these functionsmay be included in a computer readable storage medium or other computerprogram product configured for execution by one or more processors.

In some embodiments, an accessibility method is performed at anelectronic device with a touch-sensitive surface and a display. Themethod includes displaying a plurality of user interface elements on thetouch screen display wherein a current focus is on a first userinterface element. The method also detects a first finger gesture on thetouch screen display, wherein the first finger gesture is independent ofcontacting a location on the touch-sensitive surface that corresponds toa second user interface element, and in response, the method changes thecurrent focus from the first user interface element in the plurality ofuser interface elements to the second user interface element in theplurality of user interface elements, and outputs accessibilityinformation associated with the second user interface element.

In some embodiments, an accessible electronic device includes atouch-sensitive surface and a display, one or more processors, memory,and one or more programs stored in the memory and configured to beexecuted by the one or more processors. The one or more programs includeinstructions for: displaying a plurality of user interface elements onthe display wherein a current focus is on a first user interfaceelement; detecting a first finger gesture on the touch-sensitivesurface, wherein the first finger gesture is independent of contacting alocation on the touch-sensitive surface that corresponds to a seconduser interface element, and, in response to detecting the first fingergesture, changing the current focus from the first user interfaceelement in the plurality of user interface elements to the second userinterface element in the plurality of user interface elements andoutputting accessibility information associated with the second userinterface element.

In some embodiments, a computer readable storage medium has storedtherein instructions which when executed by an accessible electronicdevice with a display and a touch-sensitive surface, cause the deviceto: display a plurality of user interface elements on the displaywherein a current focus is on a first user interface element; detect afirst finger gesture on the touch-sensitive surface, wherein the firstfinger gesture is independent of contacting a location on thetouch-sensitive surface that corresponds to a second user interfaceelement; and, in response to detecting the first finger gesture, changethe current focus from the first user interface element in the pluralityof user interface elements to the second user interface element in theplurality of user interface elements and output accessibilityinformation associated with the second user interface element.

In some embodiments, a graphical user interface (“GUI”) on an accessibleelectronic device with a display and a touch-sensitive surface includesa plurality of user interface elements on the display, wherein a currentfocus is on a first user interface element. In response to detecting afirst finger gesture on the touch-sensitive surface, wherein the firstfinger gesture is independent of contacting a location on thetouch-sensitive surface that corresponds to a second user interfaceelement, the current focus is changed from the first user interfaceelement in the plurality of user interface elements to the second userinterface element in the plurality of user interface elements, andaccessibility information associated with the second user interfaceelement is outputted.

In some embodiments, an accessible electronic device includes: adisplay; a touch-sensitive surface; means for displaying a plurality ofuser interface elements on the display wherein a current focus is on afirst user interface element; means for detecting a first finger gestureon the touch-sensitive surface, wherein the first finger gesture isindependent of contacting a location on the touch-sensitive surface thatcorresponds to a second user interface element; and, in response todetecting the first finger gesture, means for changing the current focusfrom the first user interface element in the plurality of user interfaceelements to the second user interface element in the plurality of userinterface elements and means for outputting accessibility informationassociated with the second user interface element.

In some embodiments, an accessibility method is performed at anelectronic device with a display and a touch-sensitive surface. Themethod includes: displaying a first section of a document on thedisplay, wherein the document has a plurality of sections; outputting anaudible document section indicia that corresponds to the first sectionof the document; detecting a first finger gesture on the touch-sensitivesurface; and, in response to detecting the first finger gesture: ceasingto display the first section of the document; displaying a secondsection of the document on the display, wherein the second section ofthe document is adjacent to the first section of the document; andoutputting an audible document section indicia that corresponds to thesecond section of the document.

In some embodiments, an accessible electronic device includes atouch-sensitive surface and a display, one or more processors, memory,and one or more programs stored in the memory and configured to beexecuted by the one or more processors. The one or more programs includeinstructions for: displaying a first section of a document on thedisplay, wherein the document has a plurality of sections; outputting anaudible document section indicia that corresponds to the first sectionof the document; detecting a first finger gesture on the touch-sensitivesurface; and, in response to detecting the first finger gesture: ceasingto display the first section of the document, displaying a secondsection of the document on the display, wherein the second section ofthe document is adjacent to the first section of the document, andoutputting an audible document section indicia that corresponds to thesecond section of the document.

In some embodiments, a computer readable storage medium has storedtherein instructions which when executed by an accessible electronicdevice with a display and a touch-sensitive surface, cause the deviceto: display a first section of a document on the display, wherein thedocument has a plurality of sections; output an audible document sectionindicia that corresponds to the first section of the document; detect afirst finger gesture on the display, and, in response to detecting thefirst finger gesture: cease to display the first section of thedocument; display a second section of the document on the display,wherein the second section of the document is adjacent to the firstsection of the document; and output a first audible document sectionindicia that corresponds to the second section of the document.

In some embodiments, a graphical user interface (“GUI”) on an accessibleelectronic device with a display and a touch-sensitive surface includesa first section of a document, wherein the document has a plurality ofsections. In response to detecting a first finger gesture on thetouch-sensitive surface: a second section of the document is displayed,replacing the displayed first section of the document, wherein thesecond section of the document is adjacent to the first section of thedocument, and an audible document section indicia that corresponds tothe second section of the document is output.

In some embodiments, an accessible electronic device includes: atouch-sensitive surface; a display; means for displaying a first sectionof a document on the touch screen display, wherein the document has aplurality of sections; means for outputting an audible document sectionindicia that corresponds to the first section of the document; means fordetecting a first finger gesture on the touch-sensitive surface; and, inresponse to detecting the first finger gesture: means for ceasing todisplay the first section of the document; means for displaying a secondsection of the document on the display, wherein the second section ofthe document is adjacent to the first section of the document; and meansfor outputting a first audible document section indicia that correspondsto the second section of the document.

In some embodiments, a method is performed at an accessible electronicdevice with a touch-sensitive surface and a display. The methodincludes: displaying a plurality of user interface elements on thedisplay; detecting a first user interface navigation gesture by a fingeron the touch-sensitive surface; and, in response to detecting the firstuser interface navigation gesture by the finger on the touch-sensitivesurface, navigating in the plurality of user interface elements on thedisplay in accordance with a current navigable unit type, wherein thecurrent navigable unit type is set to a first navigable unit typeselected from a plurality of navigable unit types. The method alsoincludes detecting a first user interface navigation setting gesture onthe touch-sensitive surface, wherein the first user interface navigationsetting gesture is independent of contacting a location on thetouch-sensitive surface that corresponds to any of the plurality of userinterface elements; and, in response to detecting the first userinterface navigation setting gesture on the touch-sensitive surface,changing the current navigable unit type from the first navigable unittype to a second navigable unit type selected from the plurality ofnavigable unit types; and outputting accessibility information about thesecond navigable unit type. After changing the current navigable unittype from the first navigable unit type to the second navigable unittype, the method includes: detecting a second user interface navigationgesture by the finger on the touch-sensitive surface, wherein the seconduser interface navigation gesture is substantially the same as the firstuser interface navigation gesture; and, in response to detecting thesecond user interface navigation gesture by the finger on thetouch-sensitive surface, navigating in the plurality of user interfaceelements on the display in accordance with the current navigable unittype, wherein the current navigable unit type is set to the secondnavigable unit type.

In some embodiments, an accessible electronic device includes atouch-sensitive surface and a display, one or more processors, memory,and one or more programs stored in the memory and configured to beexecuted by the one or more processors. The one or more programs includeinstructions for: displaying a plurality of user interface elements onthe display; detecting a first user interface navigation gesture by afinger on the touch-sensitive surface; and, in response to detecting thefirst user interface navigation gesture by the finger on thetouch-sensitive surface, navigating in the plurality of user interfaceelements in accordance with a current navigable unit type, wherein thecurrent navigable unit type is set to a first navigable unit typeselected from a plurality of navigable unit types. The one or moreprograms also include instructions for: detecting a first user interfacenavigation setting gesture on the touch-sensitive surface, wherein thefirst user interface navigation setting gesture is independent ofcontacting a location on the touch-sensitive surface that corresponds toany of the plurality of user interface elements; in response todetecting the first user interface navigation setting gesture on thetouch-sensitive surface, changing the current navigable unit type fromthe first navigable unit type to a second navigable unit type selectedfrom the plurality of navigable unit types and outputting accessibilityinformation about the second navigable unit type; after changing thecurrent navigable unit type from the first navigable unit type to thesecond navigable unit type, detecting a second user interface navigationgesture by the finger on the touch-sensitive surface, wherein the seconduser interface navigation gesture is substantially the same as the firstuser interface navigation gesture; and, in response to detecting thesecond user interface navigation gesture by the finger on thetouch-sensitive surface, navigating in the plurality of user interfaceelements on the display in accordance with the current navigable unittype, wherein the current navigable unit type is set to the secondnavigable unit type.

In some embodiments, a computer readable storage medium has storedtherein instructions which when executed by an accessible electronicdevice with a display and a touch-sensitive surface, cause the deviceto: display a plurality of user interface elements on the display;detect a first user interface navigation gesture by a finger on thetouch-sensitive surface; in response to detecting the first userinterface navigation gesture by a finger on the touch-sensitive surface,navigate in the plurality of user interface elements in accordance witha current navigable unit type, wherein the current navigable unit typeis set to a first navigable unit type selected from a plurality ofnavigable unit types; detect a first user interface navigation settinggesture on the touch-sensitive surface, wherein the first user interfacenavigation setting gesture is independent of contacting a location onthe touch-sensitive surface that corresponds to any of the plurality ofuser interface elements; in response to detecting the first userinterface navigation setting gesture on the touch-sensitive surface:change the current navigable unit type from the first navigable unittype to a second navigable unit type selected from the plurality ofnavigable unit types and output accessibility information about thesecond navigable unit type. After changing the current navigable unittype from the first navigable unit type to the second navigable unittype, the one or more programs also comprise instructions, which whenexecuted cause the device to: detect a second user interface navigationgesture by the finger on the touch-sensitive surface, wherein the seconduser interface navigation gesture is substantially the same as the firstuser interface navigation gesture; and, in response to detecting thesecond user interface navigation gesture by the finger on thetouch-sensitive surface, navigate in the plurality of user interfaceelements in accordance with the current navigable unit type, wherein thecurrent navigable unit type is set to the second navigable unit type.

In some embodiments, a GUI on an accessible electronic device with atouch-sensitive surface and a display includes a plurality of userinterface elements on the display. In response to detecting a first userinterface navigation gesture by a finger on the touch-sensitive surface,navigation in the plurality of user interface elements occurs inaccordance with a current navigable unit type, wherein the currentnavigable unit type is set to a first navigable unit type selected froma plurality of navigable unit types. In response to detecting a firstuser interface navigation setting gesture on the touch-sensitivesurface, wherein the first user interface navigation setting gesture isindependent of contacting a location on the touch-sensitive surface thatcorresponds to any of the plurality of user interface elements: thecurrent navigable unit type is changed from the first navigable unittype to a second navigable unit type selected from the plurality ofnavigable unit types, and accessibility information about the secondnavigable unit type is outputted. After changing the current navigableunit type from the first navigable unit type to the second navigableunit type, in response to detecting a second user interface navigationgesture by the finger on the touch-sensitive surface, wherein the seconduser interface navigation gesture is substantially the same as the firstuser interface navigation gesture, navigation in the plurality of userinterface elements occurs in accordance with the current navigable unittype, wherein the current navigable unit type is set to the secondnavigable unit type.

In some embodiments, an accessible electronic device includes: atouch-sensitive surface; a display; means for displaying a plurality ofuser interface elements on the display, and means for detecting a firstuser interface navigation gesture by a finger on the touch-sensitivesurface. In response to detecting the first user interface navigationgesture by the finger on the touch-sensitive surface, the accessibleelectronic device further comprises means for navigating in theplurality of user interface elements in accordance with a currentnavigable unit type, wherein the current navigable unit type is set to afirst navigable unit type selected from a plurality of navigable unittypes, as well as means for detecting a first user interface navigationsetting gesture on the touch-sensitive surface, wherein the first userinterface navigation setting gesture is independent of contacting alocation on the touch-sensitive surface that corresponds to any of theplurality of user interface elements; in response to detecting the firstuser interface navigation setting gesture on the touch-sensitivesurface, means for changing the current navigable unit type from thefirst navigable unit type to a second navigable unit type selected fromthe plurality of navigable unit types, and means for outputtingaccessibility information about the second navigable unit type; afterchanging the current navigable unit type from the first navigable unittype to the second navigable unit type, means for detecting a seconduser interface navigation gesture by the finger on the touch-sensitivesurface, wherein the second user interface navigation gesture issubstantially the same as the first user interface navigation gesture;and, in response to detecting the second user interface navigationgesture by the finger on the touch-sensitive surface, means fornavigating in the plurality of user interface elements in accordancewith the current navigable unit type, wherein the current navigable unittype is set to the second navigable unit type.

In some embodiments, a method is performed at an accessible electronicdevice with a touch-sensitive surface and a display. The methodincludes: mapping at least a first portion of the display to thetouch-sensitive surface; concurrently displaying a plurality of userinterface containers on the display; detecting a user interfacecontainer selection event that selects a first user interface containerin the plurality of user interface containers; and, in response todetecting the user interface container selection event, ceasing to mapthe first portion of the display to the touch-sensitive surface andproportionally mapping the first user interface container to besubstantially coextensive with the touch-sensitive surface.

In some embodiments, an accessible electronic device includes atouch-sensitive surface and a display, one or more processors, memory,and one or more programs stored in the memory and configured to beexecuted by the one or more processors. The one or more programs includeinstructions for: mapping at least a first portion of the display to thetouch-sensitive surface; concurrently displaying a plurality of userinterface containers on the display; detecting a user interfacecontainer selection event that selects a first user interface containerin the plurality of user interface containers; and, in response todetecting the user interface container selection event, ceasing to mapthe first portion of the display to the touch-sensitive surface andproportionally mapping the first user interface container to besubstantially coextensive with the touch-sensitive surface.

In some embodiments, a computer readable storage medium has storedtherein instructions which when executed by an accessible electronicdevice with a display and a touch-sensitive surface, cause the deviceto: map at least a first portion of the display to the touch-sensitivesurface; concurrently display a plurality of user interface containerson the display; detect a user interface container selection event thatselects a first user interface container in the plurality of userinterface containers; and, in response to detecting the user interfacecontainer selection event, cease to map the first portion of the displayto the touch-sensitive surface and proportionally map the first userinterface container to be substantially coextensive with thetouch-sensitive surface.

In some embodiments, a GUI on an accessible electronic with a displayand a touch-sensitive surface includes a plurality of user interfacecontainers concurrently displayed on the display. At least a firstportion of the display is mapped to the touch-sensitive surface. Inresponse to detecting a user interface container selection event thatselects a first user interface container in the plurality of userinterface containers: the first portion of the display ceases to bemapped to the touch-sensitive surface, and the first user interfacecontainer is proportionally mapped to be substantially coextensive withthe touch-sensitive surface.

In some embodiments, an accessible electronic device includes: atouch-sensitive surface; a display; means for mapping at least a firstportion of the display to the touch-sensitive surface; means forconcurrently displaying a plurality of user interface containers on thedisplay; means for detecting a user interface container selection eventthat selects a first user interface container in the plurality of userinterface containers; and, in response to detecting the user interfacecontainer selection event, means for ceasing to map the first portion ofthe display to the touch-sensitive surface and means for proportionallymapping the first user interface container to be substantiallycoextensive with the touch-sensitive surface.

In some embodiments, an accessibility method is performed at anelectronic device with a touch-sensitive display having a center. Themethod includes: displaying content in a user interface at a firstmagnification on the touch screen display; detecting a firstmulti-finger gesture on the touch screen display, wherein detecting thefirst multi-finger gesture includes detecting concurrent multi-fingercontacts on the touch screen display; determining a bounding box for theconcurrent multi-finger contacts; and, in response to detecting thefirst multi-finger gesture on the touch screen display, magnifying thecontent in the user interface about a point in the bounding box to asecond magnification, greater than the first magnification, on the touchscreen display. The point in the bounding box is at a default positionwithin the bounding box when the bounding box is located at the centerof the touch screen display. When the bounding box is located away fromthe center of the touch screen display, the point in the bounding box isdisplaced from the default position within the bounding box towards arespective edge of the bounding box by an amount that corresponds to adisplacement of the bounding box from the center of the touch screendisplay towards a corresponding edge of the touch screen display.

In some embodiments, a graphical user interface on an accessibleelectronic device with a touch screen display having a center includescontent displayed at a first magnification on the touch screen display.A first multi-finger gesture is detected on the touch screen display.Detecting the first multi-finger gesture includes detecting concurrentmulti-finger contacts on the touch screen display. A bounding box forthe concurrent multi-finger contacts is determined. In response todetecting the first multi-finger gesture on the touch screen display,the content in the user interface is magnified about a point in thebounding box to a second magnification, greater than the firstmagnification, on the touch screen display. The point in the boundingbox is at a default position within the bounding box when the boundingbox is located at the center of the touch screen display. When thebounding box is located away from the center of the touch screendisplay, the point in the bounding box is displaced from the defaultposition within the bounding box towards a respective edge of thebounding box by an amount that corresponds to a displacement of thebounding box from the center of the touch screen display towards acorresponding edge of the touch screen display.

In some embodiments, an accessible electronic device includes a touchscreen display having a center, one or more processors, memory, and oneor more programs stored in the memory and configured to be executed bythe one or more processors. The one or more programs includeinstructions for: displaying content in a user interface at a firstmagnification on the touch screen display; detecting a firstmulti-finger gesture on the touch screen display, wherein detecting thefirst multi-finger gesture includes detecting concurrent multi-fingercontacts on the touch screen display; determining a bounding box for theconcurrent multi-finger contacts; and, in response to detecting thefirst multi-finger gesture on the touch screen display, magnifying thecontent in the user interface about a point in the bounding box to asecond magnification, greater than the first magnification, on the touchscreen display. The point in the bounding box is at a default positionwithin the bounding box when the bounding box is located at the centerof the touch screen display. When the bounding box is located away fromthe center of the touch screen display, the point in the bounding box isdisplaced from the default position within the bounding box towards arespective edge of the bounding box by an amount that corresponds to adisplacement of the bounding box from the center of the touch screendisplay towards a corresponding edge of the touch screen display.

In some embodiments, a computer readable storage medium has storedtherein instructions which when executed by an accessible electronicdevice with a touch screen display having a center, cause the device to:display content in a user interface at a first magnification on thetouch screen display; detect a first multi-finger gesture on the touchscreen display, wherein detecting the first multi-finger gestureincludes detecting concurrent multi-finger contacts on the touch screendisplay; determine a bounding box for the concurrent multi-fingercontacts; and, in response to detecting the first multi-finger gestureon the touch screen display, magnify the content in the user interfaceabout a point in the bounding box to a second magnification, greaterthan the first magnification, on the touch screen display. The point inthe bounding box is at a default position within the bounding box whenthe bounding box is located at the center of the touch screen display.When the bounding box is located away from the center of the touchscreen display, the point in the bounding box is displaced from thedefault position within the bounding box towards a respective edge ofthe bounding box by an amount that corresponds to a displacement of thebounding box from the center of the touch screen display towards acorresponding edge of the touch screen display.

In some embodiments, an accessible electronic device includes: a touchscreen display having a center; means for displaying content in a userinterface at a first magnification on the touch screen display; meansfor detecting a first multi-finger gesture on the touch screen display,wherein detecting the first multi-finger gesture includes detectingconcurrent multi-finger contacts on the touch screen display; means fordetermining a bounding box for the concurrent multi-finger contacts;and, in response to detecting the first multi-finger gesture on thetouch screen display, means for magnifying the content in the userinterface about a point in the bounding box to a second magnification,greater than the first magnification, on the touch screen display. Thepoint in the bounding box is at a default position within the boundingbox when the bounding box is located at the center of the touch screendisplay. When the bounding box is located away from the center of thetouch screen display, the point in the bounding box is displaced fromthe default position within the bounding box towards a respective edgeof the bounding box by an amount that corresponds to a displacement ofthe bounding box from the center of the touch screen display towards acorresponding edge of the touch screen display.

In some embodiments, an information processing apparatus for use in anaccessible electronic device is provided, wherein the accessibleelectronic device includes a touch screen display having a center. Theinformation processing apparatus includes: means for displaying contentin a user interface at a first magnification on the touch screendisplay; means for detecting a first multi-finger gesture on the touchscreen display, wherein detecting the first multi-finger gestureincludes detecting concurrent multi-finger contacts on the touch screendisplay; means for determining a bounding box for the concurrentmulti-finger contacts; and, in response to detecting the firstmulti-finger gesture on the touch screen display, means for magnifyingthe content in the user interface about a point in the bounding box to asecond magnification, greater than the first magnification, on the touchscreen display. The point in the bounding box is at a default positionwithin the bounding box when the bounding box is located at the centerof the touch screen display. When the bounding box is located away fromthe center of the touch screen display, the point in the bounding box isdisplaced from the default position within the bounding box towards arespective edge of the bounding box by an amount that corresponds to adisplacement of the bounding box from the center of the touch screendisplay towards a corresponding edge of the touch screen display.

In some embodiments, a method is performed at an accessible electronicdevice with a touch screen display. The method includes: displayingcontent in a user interface on the touch screen display; detecting aplurality of concurrent finger contacts on the touch screen display;detecting movement of the plurality of concurrent finger contacts on thetouch screen display; scrolling the content in the user interface inaccordance with the movement of the plurality of concurrent fingercontacts; detecting lift off of all but one of the plurality ofconcurrent finger contacts while continuing to detect a single remainingfinger contact; ceasing scrolling of the content in the user interfacewhen the single remaining finger contact is located in a predefinedcentral portion of the touch screen display; and scrolling the contentin the user interface when the single remaining finger contact islocated in a predefined border portion of the touch screen display.

In some embodiments, a GUI on an accessible electronic device with atouch screen display includes content displayed on the touch screen. Aplurality of concurrent finger contacts is detected on the touch screendisplay. Movement of the plurality of concurrent finger contacts on thetouch screen display is detected. The content in the user interface isscrolled in accordance with the movement of the plurality of concurrentfinger contacts. Lift off of all but one of the plurality of concurrentfinger contacts is detected while continuing to detect a singleremaining finger contact. Scrolling of the content in the user interfaceis ceased when the single remaining finger contact is located in apredefined central portion of the touch screen display. The content inthe user interface is scrolled when the single remaining finger contactis located in a predefined border portion of the touch screen display.

In some embodiments, an accessible electronic device includes a touchscreen display, one or more processors, memory, and one or more programsstored in the memory and configured to be executed by the one or moreprocessors. The one or more programs include instructions for:displaying content in a user interface on the touch screen display;detecting a plurality of concurrent finger contacts on the touch screendisplay; detecting movement of the plurality of concurrent fingercontacts on the touch screen display; scrolling the content in the userinterface in accordance with the movement of the plurality of concurrentfinger contacts; detecting lift off of all but one of the plurality ofconcurrent finger contacts while continuing to detect a single remainingfinger contact; ceasing scrolling of the content in the user interfacewhen the single remaining finger contact is located in a predefinedcentral portion of the touch screen display; and scrolling the contentin the user interface when the single remaining finger contact islocated in a predefined border portion of the touch screen display.

In some embodiments, a computer readable storage medium has storedtherein instructions which when executed by an accessible electronicdevice with a touch screen display, cause the device to: display contentin a user interface on the touch screen display; detect a plurality ofconcurrent finger contacts on the touch screen display; detect movementof the plurality of concurrent finger contacts on the touch screendisplay; scroll the content in the user interface in accordance with themovement of the plurality of concurrent finger contacts; detect lift offof all but one of the plurality of concurrent finger contacts whilecontinuing to detect a single remaining finger contact; cease scrollingof the content in the user interface when the single remaining fingercontact is located in a predefined central portion of the touch screendisplay; and scroll the content in the user interface when the singleremaining finger contact is located in a predefined border portion ofthe touch screen display.

In some embodiments, an accessible electronic device includes: a touchscreen display; means for means for displaying content in a userinterface on the touch screen display; means for detecting a pluralityof concurrent finger contacts on the touch screen display; means fordetecting movement of the plurality of concurrent finger contacts on thetouch screen display; means for scrolling the content in the userinterface in accordance with the movement of the plurality of concurrentfinger contacts; means for detecting lift off of all but one of theplurality of concurrent finger contacts while continuing to detect asingle remaining finger contact; means for ceasing scrolling of thecontent in the user interface when the single remaining finger contactis located in a predefined central portion of the touch screen display;and means for scrolling the content in the user interface when thesingle remaining finger contact is located in a predefined borderportion of the touch screen display.

In some embodiments, an information processing apparatus for use in anaccessible electronic device is provided, wherein the accessibleelectronic device includes a touch screen display. The informationprocessing apparatus includes: means for displaying content in a userinterface on the touch screen display; means for detecting a pluralityof concurrent finger contacts on the touch screen display; means fordetecting movement of the plurality of concurrent finger contacts on thetouch screen display; means for scrolling the content in the userinterface in accordance with the movement of the plurality of concurrentfinger contacts; means for detecting lift off of all but one of theplurality of concurrent finger contacts while continuing to detect asingle remaining finger contact; means for ceasing scrolling of thecontent in the user interface when the single remaining finger contactis located in a predefined central portion of the touch screen display;and means for scrolling the content in the user interface when thesingle remaining finger contact is located in a predefined borderportion of the touch screen display.

Thus, electronic devices with displays and touch-sensitive surfaces areprovided with new and improved accessibility methods and userinterfaces, thereby increasing the effectiveness, efficiency, and usersatisfaction with such devices. Such methods and interfaces maycomplement or replace existing accessibility methods.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the aforementioned embodiments of theinvention as well as additional embodiments thereof, reference should bemade to the Description of Embodiments below, in conjunction with thefollowing drawings in which like reference numerals refer tocorresponding parts throughout the figures.

FIGS. 1A and 1B are block diagrams illustrating portable multifunctiondevices with touch-sensitive displays in accordance with someembodiments.

FIG. 2 illustrates a portable multifunction device having a touch screenin accordance with some embodiments.

FIG. 3 is a block diagram of an exemplary computing device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIGS. 4A-4B illustrate exemplary user interfaces for a menu ofapplications on a portable multifunction device in accordance with someembodiments

FIG. 4C illustrates an exemplary user interface for unlocking a portableelectronic device in accordance with some embodiments.

FIGS. 5A-5QQ illustrate accessibility user interfaces for an electronicdevice with a touch-sensitive surface in accordance with someembodiments.

FIGS. 6A-6T illustrate accessibility user interfaces that map respectiveuser interface containers to a touch-sensitive surface in accordancewith some embodiments.

FIGS. 7A-7C are flow diagrams illustrating an accessibility method fortouch-based navigation among user interface elements in accordance withsome embodiments.

FIGS. 8A-8B are flow diagrams illustrating an accessibility method forcommunicating document section indicia in accordance with someembodiments.

FIGS. 9A-9B are flow diagrams illustrating an accessibility method thatuses location-independent gestures to adjust user interface navigationsettings in accordance with some embodiments.

FIGS. 10A-10C are flow diagrams illustrating an accessibility method fornavigating via user interface containers in accordance with someembodiments.

FIGS. 11A-11B are flow diagrams illustrating an accessibility method forperforming magnification about a point in a multi-finger bounding box inaccordance with some embodiments.

FIGS. 12A-12B are flow diagrams illustrating an accessibility method forsingle-finger push panning in accordance with some embodiments.

DESCRIPTION OF EMBODIMENTS

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the present invention. However, it will beapparent to one of ordinary skill in the art that the present inventionmay be practiced without these specific details. In other instances,well-known methods, procedures, components, circuits, and networks havenot been described in detail so as not to unnecessarily obscure aspectsof the embodiments.

It will also be understood that, although the terms first, second, etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another. For example, a first contact could be termed asecond contact, and, similarly, a second contact could be termed a firstcontact, without departing from the scope of the present invention. Thefirst contact and the second contact are both contacts, but they are notthe same contact.

The terminology used in the description of the invention herein is forthe purpose of describing particular embodiments only and is notintended to be limiting of the invention. As used in the description ofthe invention and the appended claims, the singular forms “a”, “an” and“the” are intended to include the plural forms as well, unless thecontext clearly indicates otherwise. Use of “indicia” may indicateeither or both of the singular usage of the term, “indicium,” or theplural form “indicia,” or vice versa.

It will also be understood that the term “and/or” as used herein refersto and encompasses any and all possible combinations of one or more ofthe associated listed items. It will be further understood that theterms “comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in response to detecting,” dependingon the context. Similarly, the phrase “if it is determined” or “if [astated condition or event] is detected” may be construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event],” depending on the context.

Embodiments of computing devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the computing device is a portable communications devicesuch as a mobile telephone that also contains other functions, such asPDA and/or music player functions. Exemplary embodiments of portablemultifunction devices include, without limitation, the iPhone® and iPodTouch® devices from Apple, Inc. of Cupertino, Calif.

In the discussion that follows, a computing device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the computing device may include one or moreother physical user interface devices, such as a physical keyboard, amouse and/or a joystick.

The device supports a variety of applications, such as one or more ofthe following: a drawing application, a presentation application, a wordprocessing application, a website creation application, a disk authoringapplication, a spreadsheet application, a gaming application, atelephone application, a video conferencing application, an e-mailapplication, an instant messaging application, a voice memo application,a photo management application, a digital camera application, a digitalvideo camera application, a web browsing application, a digital musicplayer application, and/or a digital video player application.

The various applications that may be executed on the device may use atleast one common physical user interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the device maybe adjusted and/or varied from one application to the next and/or withina respective application. In this way, a common physical architecture(such as the touch-sensitive surface) of the device may support thevariety of applications with user interfaces that are intuitive andtransparent.

The user interfaces may include one or more soft keyboard embodiments.The soft keyboard embodiments may include standard (QWERTY) and/ornon-standard configurations of symbols on the displayed icons of thekeyboard, such as those described in U.S. patent application Ser. No.11/459,606, “Keyboards For Portable Electronic Devices,” filed Jul. 24,2006, and Ser. No. 11/459,615, “Touch Screen Keyboards For PortableElectronic Devices,” filed Jul. 24, 2006, the contents of which arehereby incorporated by reference in their entirety. The keyboardembodiments may include a reduced number of icons (or soft keys)relative to the number of keys in existing physical keyboards, such asthat for a typewriter. This may make it easier for users to select oneor more icons in the keyboard, and thus, one or more correspondingsymbols. The keyboard embodiments may be adaptive. For example,displayed icons may be modified in accordance with user actions, such asselecting one or more icons and/or one or more corresponding symbols.One or more applications on the device may utilize common and/ordifferent keyboard embodiments. Thus, the keyboard embodiment used maybe tailored to at least some of the applications. In some embodiments,one or more keyboard embodiments may be tailored to a respective user.For example, one or more keyboard embodiments may be tailored to arespective user based on a word usage history (lexicography, slang,individual usage) of the respective user. Some of the keyboardembodiments may be adjusted to reduce a probability of a user error whenselecting one or more icons, and thus one or more symbols, when usingthe soft keyboard embodiments.

Attention is now directed towards embodiments of portable devices withtouch-sensitive displays. FIGS. 1A and 1B are block diagramsillustrating portable multifunction devices 100 with touch-sensitivedisplays 112 in accordance with some embodiments. The touch-sensitivedisplay 112 is sometimes called a “touch screen” for convenience, andmay also be known as or called a touch-sensitive display system. Thedevice 100 may include a memory 102 (which may include one or morecomputer readable storage mediums), a memory controller 122, one or moreprocessing units (CPU's) 120, a peripherals interface 118, RF circuitry108, audio circuitry 110, a speaker 111, a microphone 113, aninput/output (I/O) subsystem 106, other input or control devices 116,and an external port 124. The device 100 may include one or more opticalsensors 164. These components may communicate over one or morecommunication buses or signal lines 103.

It should be appreciated that the device 100 is only one example of aportable multifunction device 100, and that the device 100 may have moreor fewer components than shown, may combine two or more components, or amay have a different configuration or arrangement of the components. Thevarious components shown in FIGS. 1A and 1B may be implemented inhardware, software, or a combination of both hardware and software,including one or more signal processing and/or application specificintegrated circuits.

Memory 102 may include high-speed random access memory and may alsoinclude non-volatile memory, such as one or more magnetic disk storagedevices, flash memory devices, or other non-volatile solid-state memorydevices. Access to memory 102 by other components of the device 100,such as the CPU 120 and the peripherals interface 118, may be controlledby the memory controller 122.

The peripherals interface 118 couples the input and output peripheralsof the device to the CPU 120 and memory 102. The one or more processors120 run or execute various software programs and/or sets of instructionsstored in memory 102 to perform various functions for the device 100 andto process data.

In some embodiments, the peripherals interface 118, the CPU 120, and thememory controller 122 may be implemented on a single chip, such as achip 104. In some other embodiments, they may be implemented on separatechips.

The RF (radio frequency) circuitry 108 receives and sends RF signals,also called electromagnetic signals. The RF circuitry 108 convertselectrical signals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. The RF circuitry 108 may include well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. The RFcircuitry 108 may communicate with networks, such as the Internet, alsoreferred to as the World Wide Web (WWW), an intranet and/or a wirelessnetwork, such as a cellular telephone network, a wireless local areanetwork (LAN) and/or a metropolitan area network (MAN), and otherdevices by wireless communication. The wireless communication may useany of a plurality of communications standards, protocols andtechnologies, including but not limited to Global System for MobileCommunications (GSM), Enhanced Data GSM Environment (EDGE), high-speeddownlink packet access (HSDPA), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a,IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over InternetProtocol (VoIP), Wi-MAX, a protocol for email (e.g., Internet messageaccess protocol (IMAP) and/or post office protocol (POP)), instantmessaging (e.g., extensible messaging and presence protocol (XMPP),Session Initiation Protocol for Instant Messaging and PresenceLeveraging Extensions (SIMPLE), Instant Messaging and Presence Service(IMPS)), and/or Short Message Service (SMS)), or any other suitablecommunication protocol, including communication protocols not yetdeveloped as of the filing date of this document.

The audio circuitry 110, the speaker 111, and the microphone 113 providean audio interface between a user and the device 100. The audiocircuitry 110 receives audio data from the peripherals interface 118,converts the audio data to an electrical signal, and transmits theelectrical signal to the speaker 111. The speaker 111 converts theelectrical signal to human-audible sound waves. The audio circuitry 110also receives electrical signals converted by the microphone 113 fromsound waves. The audio circuitry 110 converts the electrical signal toaudio data and transmits the audio data to the peripherals interface 118for processing. Audio data may be retrieved from and/or transmitted tomemory 102 and/or the RF circuitry 108 by the peripherals interface 118.In some embodiments, the audio circuitry 110 also includes a headsetjack (e.g. 212, FIG. 2). The headset jack provides an interface betweenthe audio circuitry 110 and removable audio input/output peripherals,such as output-only headphones or a headset with both output (e.g., aheadphone for one or both ears) and input (e.g., a microphone).

The I/O subsystem 106 couples input/output peripherals on the device100, such as the touch screen 112 and other input/control devices 116,to the peripherals interface 118. The I/O subsystem 106 may include adisplay controller 156 and one or more input controllers 160 for otherinput or control devices. The one or more input controllers 160receive/send electrical signals from/to other input or control devices116. The other input/control devices 116 may include physical buttons(e.g., push buttons, rocker buttons, etc.), dials, slider switches,joysticks, click wheels, and so forth. In some alternate embodiments,input controller(s) 160 may be coupled to any (or none) of thefollowing: a keyboard, infrared port, USB port, and a pointer devicesuch as a mouse. The one or more buttons (e.g., 208, FIG. 2) may includean up/down button for volume control of the speaker 111 and/or themicrophone 113. The one or more buttons may include a push button (e.g.,206, FIG. 2). A quick press of the push button may disengage a lock ofthe touch screen 112 or begin a process that uses gestures on the touchscreen to unlock the device, as described in U.S. patent applicationSer. No. 11/322,549, “Unlocking a Device by Performing Gestures on anUnlock Image,” filed Dec. 23, 2005, which is hereby incorporated byreference in its entirety. A longer press of the push button (e.g., 206)may turn power to the device 100 on or off. The user may be able tocustomize a functionality of one or more of the buttons. The touchscreen 112 is used to implement virtual or soft buttons and one or moresoft keyboards.

The touch-sensitive touch screen 112 provides an input interface and anoutput interface between the device and a user. The display controller156 receives and/or sends electrical signals from/to the touch screen112. The touch screen 112 displays visual output to the user. The visualoutput may include graphics, text, icons, video, and any combinationthereof (collectively termed “graphics”). In some embodiments, some orall of the visual output may correspond to user interface objects.

A touch screen 112 has a touch-sensitive surface, sensor or set ofsensors that accepts input from the user based on haptic and/or tactilecontact. The touch screen 112 and the display controller 156 (along withany associated modules and/or sets of instructions in memory 102) detectcontact (and any movement or breaking of the contact) on the touchscreen 112 and converts the detected contact into interaction with userinterface objects (e.g., one or more soft keys, icons, web pages orimages) that are displayed on the touch screen. In an exemplaryembodiment, a point of contact between a touch screen 112 and the usercorresponds to a finger of the user.

The touch screen 112 may use LCD (liquid crystal display) technology, orLPD (light emitting polymer display) technology, although other displaytechnologies may be used in other embodiments. The touch screen 112 andthe display controller 156 may detect contact and any movement orbreaking thereof using any of a plurality of touch sensing technologiesnow known or later developed, including but not limited to capacitive,resistive, infrared, and surface acoustic wave technologies, as well asother proximity sensor arrays or other elements for determining one ormore points of contact with a touch screen 112. In an exemplaryembodiment, projected mutual capacitance sensing technology is used,such as that found in the iPhone® and iPod Touch® from Apple Computer,Inc. of Cupertino, Calif.

A touch-sensitive display in some embodiments of the touch screen 112may be analogous to the multi-touch sensitive tablets described in thefollowing U.S. Pat. No. 6,323,846 (Westerman et al.), U.S. Pat. No.6,570,557 (Westerman et al.), and/or U.S. Pat. No. 6,677,932(Westerman), and/or U.S. Patent Publication 2002/0015024A1, each ofwhich is hereby incorporated by reference in its entirety. However, atouch screen 112 displays visual output from the portable device 100,whereas touch sensitive tablets do not provide visual output.

A touch-sensitive display in some embodiments of the touch screen 112may be as described in the following applications: (1) U.S. patentapplication Ser. No. 11/381,313, “Multipoint Touch Surface Controller,”filed May 2, 2006; (2) U.S. patent application Ser. No. 10/840,862,“Multipoint Touchscreen,” filed May 6, 2004; (3) U.S. patent applicationSer. No. 10/903,964, “Gestures For Touch Sensitive Input Devices,” filedJul. 30, 2004; (4) U.S. patent application Ser. No. 11/048,264,“Gestures For Touch Sensitive Input Devices,” filed Jan. 31, 2005; (5)U.S. patent application Ser. No. 11/038,590, “Mode-Based Graphical UserInterfaces For Touch Sensitive Input Devices,” filed Jan. 18, 2005; (6)U.S. patent application Ser. No. 11/228,758, “Virtual Input DevicePlacement On A Touch Screen User Interface,” filed Sep. 16, 2005; (7)U.S. patent application Ser. No. 11/228,700, “Operation Of A ComputerWith A Touch Screen Interface,” filed Sep. 16, 2005; (8) U.S. patentapplication Ser. No. 11/228,737, “Activating Virtual Keys Of ATouch-Screen Virtual Keyboard,” filed Sep. 16, 2005; and (9) U.S. patentapplication Ser. No. 11/367,749, “Multi-Functional Hand-Held Device,”filed Mar. 3, 2006. All of these applications are incorporated byreference herein in their entirety.

The touch screen 112 may have a resolution in excess of 100 dpi. In anexemplary embodiment, the touch screen has a resolution of approximately160 dpi. The user may make contact with the touch screen 112 using anysuitable object or appendage, such as a stylus, a finger, and so forth.In some embodiments, the user interface is designed to work primarilywith finger-based contacts and gestures, which are much less precisethan stylus-based input due to the larger area of contact of a finger onthe touch screen. In some embodiments, the device translates the roughfinger-based input into a precise pointer/cursor position or command forperforming the actions desired by the user.

In some embodiments, in addition to the touch screen, the device 100 mayinclude a touchpad (not shown) for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad may be a touch-sensitive surface that is separatefrom the touch screen 112 or an extension of the touch-sensitive surfaceformed by the touch screen.

In some embodiments, the device 100 may include a physical or virtualclick wheel as an input control device 116. A user may navigate amongand interact with one or more graphical objects (e.g., icons) displayedin the touch screen 112 by rotating the click wheel or by moving a pointof contact with the click wheel (e.g., where the amount of movement ofthe point of contact is measured by its angular displacement withrespect to a center point of the click wheel). The click wheel may alsobe used to select one or more of the displayed icons. For example, theuser may press down on at least a portion of the click wheel or anassociated button. User commands and navigation commands provided by theuser via the click wheel may be processed by an input controller 160 aswell as one or more of the modules and/or sets of instructions in memory102. For a virtual click wheel, the click wheel and click wheelcontroller may be part of the touch screen 112 and the displaycontroller 156, respectively. For a virtual click wheel, the click wheelmay be either an opaque or semitransparent object that appears anddisappears on the touch screen display in response to user interactionwith the device. In some embodiments, a virtual click wheel is displayedon the touch screen of a portable multifunction device and operated byuser contact with the touch screen.

The device 100 also includes a power system 162 for powering the variouscomponents. The power system 162 may include a power management system,one or more power sources (e.g., battery, alternating current (AC)), arecharging system, a power failure detection circuit, a power converteror inverter, a power status indicator (e.g., a light-emitting diode(LED)) and any other components associated with the generation,management and distribution of power in portable devices.

The device 100 may also include one or more optical sensors 164. FIGS.1A and 1B show an optical sensor coupled to an optical sensor controller158 in I/O subsystem 106. The optical sensor 164 may includecharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. The optical sensor 164 receives light from theenvironment, projected through one or more lens, and converts the lightto data representing an image. In conjunction with an imaging module 143(also called a camera module), the optical sensor 164 may capture stillimages or video. In some embodiments, an optical sensor is located onthe back of the device 100, opposite the touch screen display 112 on thefront of the device, so that the touch screen display may be used as aviewfinder for still and/or video image acquisition. In someembodiments, an optical sensor is located on the front of the device sothat the user's image may be obtained for videoconferencing while theuser views the other video conference participants on the touch screendisplay. In some embodiments, the position of the optical sensor 164 canbe changed by the user (e.g., by rotating the lens and the sensor in thedevice housing) so that a single optical sensor 164 may be used alongwith the touch screen display for both video conferencing and stilland/or video image acquisition.

The device 100 may also include one or more proximity sensors 166. FIGS.1A and 1B show a proximity sensor 166 coupled to the peripheralsinterface 118. Alternately, the proximity sensor 166 may be coupled toan input controller 160 in the I/O subsystem 106. The proximity sensor166 may perform as described in U.S. patent application Ser. No.11/241,839, “Proximity Detector In Handheld Device”; Ser. No.11/240,788, “Proximity Detector In Handheld Device”; Ser. No.11/620,702, “Using Ambient Light Sensor To Augment Proximity SensorOutput”; Ser. No. 11/586,862, “Automated Response To And Sensing Of UserActivity In Portable Devices”; and Ser. No. 11/638,251, “Methods AndSystems For Automatic Configuration Of Peripherals,” which are herebyincorporated by reference in their entirety. In some embodiments, theproximity sensor turns off and disables the touch screen 112 when themultifunction device is placed near the user's ear (e.g., when the useris making a phone call). In some embodiments, the proximity sensor keepsthe screen off when the device is in the user's pocket, purse, or otherdark area to prevent unnecessary battery drainage when the device is alocked state.

The device 100 may also include one or more accelerometers 168. FIGS. 1Aand 1B show an accelerometer 168 coupled to the peripherals interface118. Alternately, the accelerometer 168 may be coupled to an inputcontroller 160 in the I/O subsystem 106. The accelerometer 168 mayperform as described in U.S. Patent Publication No. 20050190059,“Acceleration-based Theft Detection System for Portable ElectronicDevices,” and U.S. Patent Publication No. 20060017692, “Methods AndApparatuses For Operating A Portable Device Based On An Accelerometer,”both of which are which are incorporated by reference herein in theirentirety. In some embodiments, information is displayed on the touchscreen display in a portrait view or a landscape view based on ananalysis of data received from the one or more accelerometers.

In some embodiments, the software components stored in memory 102 mayinclude an operating system 126, a communication module (or set ofinstructions) 128, an accessibility module 129, a contact/motion module(or set of instructions) 130, a graphics module (or set of instructions)132, an attachment editing module 133, a text input module (or set ofinstructions) 134, a Global Positioning System (GPS) module (or set ofinstructions) 135, and applications (or set of instructions) 136.

The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X,WINDOWS, or an embedded operating system such as VxWorks) includesvarious software components and/or drivers for controlling and managinggeneral system tasks (e.g., memory management, storage device control,power management, etc.) and facilitates communication between varioushardware and software components.

The communication module 128 facilitates communication with otherdevices over one or more external ports 124 and also includes varioussoftware components for handling data received by the RF circuitry 108and/or the external port 124. The external port 124 (e.g., UniversalSerial Bus (USB), FIREWIRE, etc.) is adapted for coupling directly toother devices or indirectly over a network (e.g., the Internet, wirelessLAN, etc.). In some embodiments, the external port is a multi-pin (e.g.,30-pin) connector that is the same as, or similar to and/or compatiblewith the 30-pin connector used on iPod (trademark of Apple Computer,Inc.) devices.

In conjunction with audio circuitry 110, speaker 111, touch screen 112,display controller 156, contact module 130, graphics module 132, andtext input module 134, the accessibility module 129 facilitatestouch-based navigation among user interface elements so that a user maynavigate, select, activate, and otherwise interact with elements in theuser interface without necessarily seeing the user interface. In someembodiments, the accessibility module 129 facilitates selecting andactivating user interface elements within the user interface withoutdirectly selecting or contacting those user interface elements.Exemplary user interface elements include, without limitation, userinterface icons and widgets, application icons, application interfaces,menus, web browsers, web pages and applications from the world-wide web,application controls, documents, soft/virtual keyboards and numericpads, calculators, calendars, lists, tables, emails, HTML text, XMLtext, rich text, unformatted text, maps, game interfaces, etc. Userinterface elements include any aspect of a graphical or textual userinterface that a user may interact with or manipulate when using anelectronic device the user interface is running on.

The contact/motion module 130 may detect contact with the touch screen112 (in conjunction with the display controller 156) and other touchsensitive devices (e.g., a touchpad or physical click wheel). Thecontact/motion module 130 includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining if there is movement of the contact and tracking themovement across the touch-sensitive surface (e.g., detecting one or morefinger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact). Thecontact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, may include determining speed(magnitude), velocity (magnitude and direction), and/or an acceleration(a change in magnitude and/or direction) of the point of contact. Theseoperations may be applied to single contacts (e.g., one finger contacts)or to multiple simultaneous contacts (e.g., “multitouch”/multiple fingercontacts). In some embodiments, the contact/motion module 130 and thedisplay controller 156 detects contact on a touchpad. In someembodiments, the contact/motion module 130 and the controller 160detects contact on a click wheel.

The contact/motion module 130 may detect a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns. Thus, a gesture may be detected by detecting a particularcontact pattern. For example, detecting a finger tap gesture comprisesdetecting a finger-down event followed by detecting a finger-up event atthe same position (or substantially the same position) as thefinger-down event (e.g., at the position of an icon). As anotherexample, detecting a finger swipe gesture on the touch-sensitive surfacecomprises detecting a finger-down event followed by detecting one ormore finger-dragging events, and subsequently followed by detecting afinger-up event.

The graphics module 132 includes various known software components forrendering and displaying graphics on the touch screen 112 or otherdisplay, including components for changing the intensity of graphicsthat are displayed. As used herein, the term “graphics” includes anyobject that can be displayed to a user, including without limitationtext, web pages, icons (such as user interface objects including softkeys), digital images, videos, animations and the like.

In some embodiments, the graphics module 132 stores data representinggraphics to be used. Each graphic may be assigned a corresponding code.The graphics module 132 receives, from applications etc., one or morecodes specifying graphics to be displayed along with, if necessary,coordinate data and other graphic property data, and then generatesscreen image data to output to display controller 156.

The text input module 134, which may be a component of graphics module132, provides soft keyboards for entering text in various applications(e.g., contacts 137, e-mail 140, IM 141, browser 147, and any otherapplication that needs text input).

The GPS module 135 determines the location of the device and providesthis information for use in various applications (e.g., to telephone 138for use in location-based dialing, to camera 143 as picture/videometadata, and to applications that provide location-based services suchas weather widgets, local yellow page widgets, and map/navigationwidgets).

The applications 136 may include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   a contacts module 137 (sometimes called an address book or        contact list);    -   a telephone module 138;    -   a video conferencing module 139;    -   an e-mail client module 140;    -   an instant messaging (IM) module 141;    -   a voice memo module 142;    -   a camera module 143 for still and/or video images;    -   an image management module 144;    -   a video player module 145;    -   a music player module 146;    -   a browser module 147;    -   a calendar module 148;    -   widget modules 149, which may include weather widget 149-1,        stocks widget 149-2, calculator widget 149-3, alarm clock widget        149-4, dictionary widget 149-5, and other widgets obtained by        the user, as well as user-created widgets 149-6;    -   widget creator module 150 for making user-created widgets 149-6;    -   search module 151;    -   video and music player module 152, which merges video player        module 145 and music player module 146;    -   notes module 153;    -   map module 154; and/or    -   online video module 155.

Examples of other applications 136 that may be stored in memory 102include other word processing applications, other image editingapplications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, and text input module 134, the contactsmodule 137 may be used to manage an address book or contact list,including: adding name(s) to the address book; deleting name(s) from theaddress book; associating telephone number(s), e-mail address(es),physical address(es) or other information with a name; associating animage with a name; categorizing and sorting names; providing telephonenumbers or e-mail addresses to initiate and/or facilitate communicationsby telephone 138, video conference 139, e-mail 140, or IM 141; and soforth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact module130, graphics module 132, and text input module 134, the telephonemodule 138 may be used to enter a sequence of characters correspondingto a telephone number, access one or more telephone numbers in theaddress book 137, modify a telephone number that has been entered, diala respective telephone number, conduct a conversation and disconnect orhang up when the conversation is completed. As noted above, the wirelesscommunication may use any of a plurality of communications standards,protocols and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, optical sensor164, optical sensor controller 158, contact module 130, graphics module132, text input module 134, contact list 137, and telephone module 138,the videoconferencing module 139 may be used to initiate, conduct, andterminate a video conference between a user and one or more otherparticipants.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, the e-mail client module 140 may be used to create, send,receive, and manage e-mail. In conjunction with image management module144, the e-mail module 140 makes it very easy to create and send e-mailswith still or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, the instant messaging module 141 may be used to enter asequence of characters corresponding to an instant message, to modifypreviously entered characters, to transmit a respective instant message(for example, using a Short Message Service (SMS) or Multimedia MessageService (MMS) protocol for telephony-based instant messages or usingXMPP, SIMPLE, or IMPS for Internet-based instant messages), to receiveinstant messages and to view received instant messages. In someembodiments, transmitted and/or received instant messages may includegraphics, photos, audio files, video files and/or other attachments asare supported in a MMS and/or an Enhanced Messaging Service (EMS). Asused herein, “instant messaging” refers to both telephony-based messages(e.g., messages sent using SMS or MMS) and Internet-based messages(e.g., messages sent using XMPP, SIMPLE, or IMPS).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, text inputmodule 134, e-mail client module 140 and instant messaging module 141,the voice memo module 142 may be used to record audio of lectures,dictation, telephone calls, conversations, performances, etc., and sendthe audio in an email or instant message.

In conjunction with touch screen 112, display controller 156, opticalsensor(s) 164, optical sensor controller 158, contact module 130,graphics module 132, and image management module 144, the camera module143 may be used to capture still images or video (including a videostream) and store them into memory 102, modify characteristics of astill image or video, or delete a still image or video from memory 102.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, text input module 134, and cameramodule 143, the image management module 144 may be used to arrange,modify (e.g., edit), or otherwise manipulate, label, delete, present(e.g., in a digital slide show or album), and store still and/or videoimages.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, audio circuitry 110, and speaker 111,the video player module 145 may be used to display, present or otherwiseplay back videos (e.g., on the touch screen or on an external, connecteddisplay via external port 124).

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, audio circuitry 110, speaker111, RF circuitry 108, and browser module 147, the music player module146 allows the user to download and play back recorded music and othersound files stored in one or more file formats, such as MP3 or AACfiles. In some embodiments, the device 100 may include the functionalityof an MP3 player, such as an iPod (trademark of Apple Computer, Inc.).

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, and text inputmodule 134, the browser module 147 may be used to browse the Internet,including searching, linking to, receiving, and displaying web pages orportions thereof, as well as attachments and other files linked to webpages.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, e-mail module 140, and browser module 147, the calendarmodule 148 may be used to create, display, modify, and store calendarsand data associated with calendars (e.g., calendar entries, to do lists,etc.).

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, and browser module 147, the widget modules 149 aremini-applications that may be downloaded and used by a user (e.g.,weather widget 149-1, stocks widget 149-2, calculator widget 149-3,alarm clock widget 149-4, and dictionary widget 149-5) or created by theuser (e.g., user-created widget 149-6). In some embodiments, a widgetincludes an HTML (Hypertext Markup Language) file, a CSS (CascadingStyle Sheets) file, and a JavaScript file. In some embodiments, a widgetincludes an XML (Extensible Markup Language) file and a JavaScript file(e.g., Yahoo! Widgets).

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, and browser module 147, the widget creator module 150 may beused by a user to create widgets (e.g., turning a user-specified portionof a web page into a widget).

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, and text input module 134, thesearch module 151 may be used to search for text, music, sound, image,video, and/or other files in memory 102 that match one or more searchcriteria (e.g., one or more user-specified search terms).

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, and text input module 134, the notesmodule 153 may be used to create and manage notes, to do lists, and thelike.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, GPS module 135, and browser module 147, the map module 154may be used to receive, display, modify, and store maps and dataassociated with maps (e.g., driving directions; data on stores and otherpoints of interest at or near a particular location; and otherlocation-based data).

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, audio circuitry 110, speaker111, RF circuitry 108, text input module 134, e-mail client module 140,and browser module 147, the online video module 155 allows the user toaccess, browse, receive (e.g., by streaming and/or download), play back(e.g., on the touch screen or on an external, connected display viaexternal port 124), send an e-mail with a link to a particular onlinevideo, and otherwise manage online videos in one or more file formats,such as H.264. In some embodiments, instant messaging module 141, ratherthan e-mail client module 140, is used to send a link to a particularonline video. Additional description of the online video application canbe found in U.S. Provisional Patent Application No. 60/936,562,“Portable Multifunction Device, Method, and Graphical User Interface forPlaying Online Videos,” filed Jun. 20, 2007, and U.S. patent applicationSer. No. 11/968,067, “Portable Multifunction Device, Method, andGraphical User Interface for Playing Online Videos,” filed Dec. 31,2007, the content of which is hereby incorporated by reference in itsentirety.

Each of the above identified modules and applications correspond to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. For example, video player module 145may be combined with music player module 146 into a single module (e.g.,video and music player module 152, FIG. 1B). In some embodiments, memory102 may store a subset of the modules and data structures identifiedabove. Furthermore, memory 102 may store additional modules and datastructures not described above.

In some embodiments, the device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen 112 and/or a touchpad. By using a touch screenand/or a touchpad as the primary input/control device for operation ofthe device 100, the number of physical input/control devices (such aspush buttons, dials, and the like) on the device 100 may be reduced.

The predefined set of functions that may be performed exclusivelythrough a touch screen and/or a touchpad include navigation between userinterfaces. In some embodiments, the touchpad, when touched by the user,navigates the device 100 to a main, home, or root menu from any userinterface that may be displayed on the device 100. In such embodiments,the touchpad may be referred to as a “menu button.” In some otherembodiments, the menu button may be a physical push button or otherphysical input/control device instead of a touchpad.

FIG. 2 illustrates a portable multifunction device 100 having a touchscreen 112 in accordance with some embodiments. The touch screen maydisplay one or more graphics within user interface (UI) 200. In thisembodiment, as well as others described below, a user may select one ormore of the graphics by making contact or touching the graphics, forexample, with one or more fingers 202 (not drawn to scale in thefigure). In some embodiments, selection of one or more graphics occurswhen the user breaks contact with the one or more graphics. In someembodiments, the contact may include a gesture, such as one or moretaps, one or more swipes (from left to right, right to left, upwardand/or downward) and/or a rolling of a finger (from right to left, leftto right, upward and/or downward) that has made contact with the device100. In some embodiments, inadvertent contact with a graphic may notselect the graphic. For example, a swipe gesture that sweeps over anapplication icon may not select the corresponding application when thegesture corresponding to selection is a tap.

The device 100 may also include one or more physical buttons, such as“home” or menu button 204. As described previously, the menu button 204may be used to navigate to any application 136 in a set of applicationsthat may be executed on the device 100. Alternatively, in someembodiments, the menu button is implemented as a soft key in a GUI intouch screen 112.

In one embodiment, the device 100 includes a touch screen 112, a menubutton 204, a push button 206 for powering the device on/off and lockingthe device, volume adjustment button(s) 208, a Subscriber IdentityModule (SIM) card slot 210, a head set jack 212, and a docking/chargingexternal port 124. The push button 206 may be used to turn the poweron/off on the device by depressing the button and holding the button inthe depressed state for a predefined time interval; to lock the deviceby depressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In an alternative embodiment, the device 100 also mayaccept verbal input for activation or deactivation of some functionsthrough the microphone 113.

FIG. 3 is a block diagram of an exemplary computing device with adisplay and a touch-sensitive surface in accordance with someembodiments. Device 300 need not be portable. In some embodiments, thedevice 300 is a laptop computer, a desktop computer, a table computer, amultimedia player device, a navigation device, an educational device(such as a child's learning toy), a gaming system, or a control device(e.g., a home or industrial controller). The device 300 typicallyincludes one or more processing units (CPU's) 310, one or more networkor other communications interfaces 360, memory 370, and one or morecommunication buses 320 for interconnecting these components. Thecommunication buses 320 may include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents. The device 300 includes an input/output (I/O) interface 330comprising a display 340, which in some embodiments is a touch screendisplay 112. The I/O interface 330 also may include a keyboard and/ormouse (or other pointing device) 350 and a touchpad 355. Memory 370includes high-speed random access memory, such as DRAM, SRAM, DDR RAM orother random access solid state memory devices; and may includenon-volatile memory, such as one or more magnetic disk storage devices,optical disk storage devices, flash memory devices, or othernon-volatile solid state storage devices. Memory 370 may optionallyinclude one or more storage devices remotely located from the CPU(s)310. In some embodiments, memory 370 stores programs, modules, and datastructures analogous to the programs, modules, and data structuresstored in the memory 102 of portable multifunction device 100 (FIG. 1),or a subset thereof. Furthermore, memory 370 may store additionalprograms, modules, and data structures not present in the memory 102 ofportable multifunction device 100. For example, memory 370 of device 300may store drawing module 380, presentation module 382, word processingmodule 384, website creation module 386, disk authoring module 388,spreadsheet module 390 and/or attachment editing module 133, whilememory 102 of portable multifunction device 100 (FIG. 1) may not storethese modules.

Each of the above identified elements in FIG. 3 may be stored in one ormore of the previously mentioned memory devices. Each of the aboveidentified modules corresponds to a set of instructions for performing afunction described above. The above identified modules or programs(i.e., sets of instructions) need not be implemented as separatesoftware programs, procedures or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. In some embodiments, memory 370 may store a subset of themodules and data structures identified above. Furthermore, memory 370may store additional modules and data structures not described above.

Attention is now directed towards embodiments of user interfaces (“UI”)that may be implemented on a portable multifunction device 100.

FIGS. 4A and 4B illustrate exemplary user interfaces for a menu ofapplications on a portable multifunction device 100 in accordance withsome embodiments. Similar user interfaces may be implemented on device300. In some embodiments, user interface 400A includes the followingelements, or a subset or superset thereof:

-   -   Signal strength indicator(s) 402 for wireless communication(s),        such as cellular and Wi-Fi signals;    -   Time 404;    -   Bluetooth indicator 405;    -   Battery status indicator 406;    -   Tray 408 with icons for frequently used applications, such as:        -   Phone 138, which may include an indicator 414 of the number            of missed calls or voicemail messages;        -   E-mail client 140, which may include an indicator 410 of the            number of unread e-mails;        -   Browser 147; and        -   Music player 146; and    -   Icons for other applications, such as:        -   IM 141;        -   Image management 144;        -   Camera 143;        -   Video player 145;        -   Weather 149-1;        -   Stocks 149-2;        -   Voice Memo 142;        -   Calendar 148;        -   Calculator 149-3;        -   Alarm clock 149-4;        -   Dictionary 149-5; and        -   User-created widget 149-6.

In some embodiments, user interface 400B includes the followingelements, or a subset or superset thereof:

-   -   402, 404, 405, 406, 141, 148, 144, 143, 149-3, 149-2, 149-1,        149-4, 410, 414, 138, 140, and 147, as described above;    -   Map 154;    -   Notes 153;    -   Settings 412, which provides access to settings for the device        100 and its various applications 136, as described further        below;    -   Video and music player module 152, also referred to as iPod        (trademark of Apple Computer, Inc.) module 152; and    -   Online video module 155, also referred to as YouTube (trademark        of Google, Inc.) module 155.

FIG. 4C illustrates an exemplary user interface for unlocking a portableelectronic device in accordance with some embodiments. In someembodiments, user interface 400C includes the following elements, or asubset or superset thereof:

-   -   Unlock image 450 that is moved with a finger gesture to unlock        the device;    -   Arrow 455 that provides a visual cue to the unlock gesture;    -   Channel 460 that provides additional cues to the unlock gesture;    -   Time 465;    -   Day 466;    -   Date 467; and    -   Wallpaper image 470.

In some embodiments, the device detects contact with the touch-sensitivedisplay (e.g., a user's finger making contact on or near the unlockimage 450) while the device is in a user interface lock state. Thedevice moves the unlock image 450 in accordance with the contact. Thedevice transitions to a user interface unlock state if the detectedcontact corresponds to a predefined gesture, such as moving the unlockimage across channel 460. Conversely, the device maintains the userinterface lock state if the detected contact does not correspond to thepredefined gesture. As noted above, processes that use gestures on thetouch screen to unlock the device are described in U.S. patentapplication Ser. No. 11/322,549, “Unlocking A Device By PerformingGestures On An Unlock Image,” filed Dec. 23, 2005, and Ser. No.11/322,550, “Indication Of Progress Towards Satisfaction Of A User InputCondition,” filed Dec. 23, 2005, which are hereby incorporated byreference in their entirety.

Attention is now directed towards exemplary embodiments of userinterfaces (“UI”) and associated processes that may be implemented on acomputing device with a display and a touch-sensitive surface, such asdevice 300 or portable multifunction device 100.

FIGS. 5A-5NN illustrate accessibility user interfaces for an electronicdevice with a touch-sensitive surface in accordance with someembodiments. In these figures, gestures depicted on the user interfaces(e.g., gestures depicted in UI 500A, UI 500K, UI 500P, and UI 500U) maybe performed by a user on a touch screen display, a track pad, or othertouch-sensitive surface. That the exemplary gestures are illustrated inthe figures on the user interfaces does not require that the gestures beperformed on a touch screen display to practice the methods disclosedherein. For example, in some embodiments, the gestures may be performedon a track pad instead. The accessibility user interfaces in thesefigures are used to illustrate processes described later in thisdisclosure.

Audible Navigation

The exemplary user interfaces depicted in FIGS. 5A-5NN and 6A-6T are foraccessibility interfaces for touch-based navigation among user interfaceelements on an electronic device. The exemplary user interfaces depictedin FIGS. 5A-5NN and 6A-6T include a “current focus,” which is notvisually depicted. Many of these exemplary user interfaces also includea visually displayed “accessibility cursor.” The accessibility cursorindicates where the current focus is. The current focus is associatedwith a specific user interface element, which, if selected, would causethe device to execute an action associated with that specific userinterface element. For example, in UI 500A (FIG. 5A) both the displayedaccessibility cursor 502 and the current focus are on or at the textingicon 141, so if that icon were selected, the device running the userinterface would execute the instant messaging application associatedwith icon 141.

Although in some embodiments an accessibility cursor is displayedvisually in the user interface, the accessibility cursor is notdisplayed visually in other embodiments. Indeed, in some embodiments,there is no requirement that a user interface is visually displayed atall. In those embodiments, audible and/or tactile feedback (e.g.,Braille keyboards) which may be communicatively attached to or disposedin the device, is provided to a user so that the user may performtouch-based navigation among non-displayed user interface elementsaccording to methods disclosed herein. In the exemplary user interfacesin FIGS. 5A-5NN, however, the current focus is at or on the same userinterface element where the accessibility cursor 502 is positioned (whenthe accessibility cursor is displayed).

Further, in some embodiments, some or all of the exemplary gesturescontained in Table 1 may be utilized in accessibility interfaces as setforth herein. In Table 1, a gesture on the “touch-sensitive surface”refers to gestures performed on a touch screen display, a track pad, orother touch-sensitive surface.

TABLE 1 Exemplary accessibility gestures Detected Gesture Response toDetected Gesture Navigation and Reading Single-finger tap on the touch-Speak the name of the user interface element sensitive surface. or itemwhere the current focus is set. Single-finger flick right on the touch-Move the current focus to the next user sensitive surface. interfaceelement. Single-finger flick left on the touch- Move the current focusto the previous user sensitive surface. interface element. Single-fingerflick down on the touch- Output audible indicia concerning the usersensitive surface. interface element where the current focus is set andadvance the current focus by one navigable unit. For example: read thename of an element and advance the current focus to the next element;speak a word and advance the current focus to the next word; or speak anindividual character in the element and advance the current focus to thenext character in the element. Single-finger flick up on the touch-Output audible indicia concerning the user sensitive surface. interfaceelement where the current focus is set and move the current focusbackward by one navigable unit. For example: read the name of an elementand move the current focus to the previous element; speak a word andmove the current focus to the previous word; or speak an individualcharacter in the element and move the current focus to the previouscharacter in the element. Two-finger single tap on the touch- Toggleoutput of current accessibility sensitive surface. information, e.g.,toggle between pausing and playing audible accessibility information.Two-finger flick upward on the touch- Read all displayed informationfrom the top sensitive surface, i.e., a two-finger flick of the display(e.g., text and/or names of toward the top of the touch-sensitivesurface. user interface elements). Two-finger flick downward on thetouch- Read all displayed information from the sensitive surface, i.e.,a two-finger flick location of the current focus on the display towardthe bottom of the touch-sensitive (e.g., text and/or names of userinterface surface. elements). Three-finger flick upward on the touch-Scroll forward/upward one page/screen of sensitive surface, i.e., athree-finger flick material, e.g., a document or list slides up ontoward the top of the touch-sensitive surface. the display, revealing anext page/screen of material. Three-finger flick downward on the touch-Scroll backward/downward one page/screen sensitive surface, i.e., athree-finger flick of material, e.g., a document or list slides towardthe bottom of the touch-sensitive down on the display, revealing aprevious surface. page/screen of material. Three-finger flick right onthe touch- Display the previous page/screen of user sensitive surface.interface elements. Three-finger flick left on the touch- Display thenext page/screen of user sensitive surface. interface elements.Three-finger single tap on the touch- Output audible document sectionindicia that sensitive surface. correspond to the currently displayedsection of the document e.g., spoken words that correspond to thedisplayed document section, such as “page 2 of 22.” Three-finger doubletap on the touch- Toggle between providing audible output sensitivesurface. and muting the audible output. Selection and ActivationSingle-finger double tap on the touch- Perform an action associated withthe user sensitive surface. interface element that has the current focus(e.g., select a key on a keyboard for text entry; activate a selecteditem; run an application, etc.). Split tapping, i.e., while touching anitem (or Perform an action associated with the a point corresponding tothe item) with one touched item (which has the current focus as finger,tap the touch-sensitive surface with a result of the touch) anotherfinger. Single-finger double tap, with the second tap Enablepass-through of gestures to a remaining in contact with thetouch-sensitive standard, non-accessibility user interface. In surfacefor a predefined period, e.g., a period some embodiments, select aspecial of time between 0.05 and 1.5 seconds. character from a group ofspecial characters, e.g., selecting accented characters such as aSpanish-language ñ. Two-finger double tap on the touch- Depending onapplication(s) running: sensitive surface. Answer or end a telephonecall. Play or pause media content playback. Take a picture with acamera. Start or pause recording with an application such as a videocamera or an audio recording tool. Manage accepting and rejectingcompletion candidates, e.g., completion candidate words suggested whentyping, or completion candidates for selecting Asian-language characterssuch as Kanji. Two-finger triple tap on the touch- Toggle output ofaccessibility information sensitive surface. on and off, e.g., mute orspeak audible accessibility information. Three-finger triple tap on thetouch- Toggle a privacy screen on and off, i.e., sensitive surface.toggle display of a “screen curtain” or blank screen that preventsviewing of displayed content and user interface elements.

The accessibility gestures in Table 1 are merely exemplary. In someembodiments, the responses to opposite gestures (e.g., a rightwardgesture versus the corresponding leftward gesture, or an upward gestureversus the corresponding downward gesture) may be reversed from thoseshown in Table 1. For example, a single-finger flick right may move thecurrent focus to the previous element and a single-finger flick left maymove the current focus to the next element. In some embodiments, theresponses to opposite gestures are user configurable, e.g., via asettings or options menu.

In some embodiments, the accessibility user interface allows a user toassociate a command with a predefined gesture. In some embodiments, agesture may be combined with a modifier key (e.g., a control, option,command, or shift key) and mapped to any command of the user's choice.For example, a user may choose to map a two-finger rotation gesture withthe control key activated to a command that adjusts the speaking rate ofthe accessibility information.

UI 500A-UI 500G (FIGS. 5A-5G) depict an example of advancing through asequence of user interface elements displayed on one user interfacescreen.

UI 500A (FIG. 5A) illustrates an exemplary user interface for a menu ofapplications on a portable multifunction device 100, where the userinterface includes an accessibility cursor 502 surrounding anapplication icon, here, texting icon 141. The accessibility cursor 502is at texting icon 141, so the current focus is at or on texting icon141.

User gesture 504, e.g., a finger swipe or flick gesture, has an initialcontact 504-1 on the touch screen 112 that moves 504-2 towards the rightedge of the touch screen 112. Note that in UI 500A, exemplary usergesture 504 is independent of contacting texting icon 141, photosapplication icon 144, or any other icons displayed in the accessibilityuser interface.

UI 500B (FIG. 5B) illustrates the exemplary user interface followinguser gesture 504. The accessibility cursor 502 has moved from textingicon 141 to photos application icon 144 in response to user gesture 504.This indicates that the current focus is now at or on photos applicationicon 144, rather than texting icon 141. Audible information associatedwith the photos application icon 144, such as “photos” or “photosapplication” or “tap to select photos”, is also output in response touser gesture 504.

UI 500C (FIG. 5C) depicts user gesture 506 on the exemplary userinterface, e.g., another finger swipe or flick gesture. Gesture 506 hasan initial contact 506-1 on the touch screen 112 that moves 506-2towards the right edge of the touch screen 112. Note that exemplary usergesture 506 remains independent of contacting photos application icon144, camera application icon 143, or any other icons displayed in theaccessibility user interface.

UI 500D (FIG. 5D) depicts that, in response to user gesture 506 (FIG.5C), the current focus, and thus, the accessibility cursor 502 has movedto camera application icon 143. Audible information associated with thecamera application icon 143, such as “camera” or “camera application” or“tap to select camera”, is also output in response to user gesture 506.

UI 500D also depicts user gesture 508 on the exemplary user interface,e.g., another finger swipe or flick gesture. Gesture 508 has an initialcontact 508-1 on the touch screen 112 that moves 508-2 towards the rightedge of the touch screen 112. Note that exemplary user gesture 508remains independent of contacting camera application icon 143, videosapplication icon 145, or any other icons displayed in the accessibilityuser interface.

UI 500E (FIG. 5E) depicts that, in response to user gesture 508 (FIG.5D), the current focus, and thus, the accessibility cursor 502 has movedto videos application icon 145. Audible information associated with thevideos application icon 145, such as “videos” or “videos application” or“tap to select videos”, is also output in response to user gesture 508.

UI 500E also depicts user gesture 510 on the exemplary user interface,e.g., another finger swipe or flick gesture. Gesture 510 has an initialcontact 510-1 on the touch screen 112 that moves 510-2 towards the rightedge of the touch screen 112. Note that exemplary user gesture 510 doesincidentally contact weather application icon 149-1, but the detection,interpretation, and response to gesture 510 remains independent ofcontacting videos application icon 145, weather application icon 149-1,or any other icons displayed in the accessibility user interface.

UI 500F (FIG. 5F) depicts that, in response to user gesture 510 (FIG.5E), the current focus, and thus, the accessibility cursor 502 has movedfrom a first row of application icons 514 to the weather applicationicon 149-1 in the next row of application icons 516. Audible informationassociated with the weather application icon 149-1, such as “weather” or“weather application” or “tap to select weather”, is also output inresponse to user gesture 510. In some embodiments, the audibleinformation also includes a wrapping sound to indicate that the currentfocus has moved to a user interface element in another row in responseto user gesture 510. A user may continue to navigate from left-to-right,top-to-bottom through the set of user interface elements in theaccessibility user interface by continuing to input left-to-right fingerswipe or flick gestures (not shown).

UI 500F also depicts user gesture 512 on the exemplary user interface,e.g., a right-to-left finger swipe or flick gesture. Gesture 512, whichis substantially opposite the direction of user gestures 504, 506, 508,and 510, has an initial contact 512-1 on the touch screen 112 that moves512-2 towards the left edge of the touch screen 112. Note that exemplaryuser gesture 512 remains independent of contacting weather applicationicon 149-1, videos application icon 145, or any other icons displayed inthe accessibility user interface.

UI 500G (FIG. 5G) depicts that, in response to user gesture 512 (FIG.5F), the current focus, and thus, the accessibility cursor 502 has movedback to videos application icon 145 in the first row of applications514. Audible information associated with the videos application icon145, such as “videos” or “videos application” or “tap to select videos”,is also output in response to user gesture 512. In some embodiments, theaudible information also includes a wrapping sound to indicate that thecurrent focus has moved to a user interface element in another row inresponse to user gesture 512. Accordingly, location-independent usergestures in one direction may advance the current focus through a givensequence or order of user interface elements; while location-independentuser gestures in a substantially opposite direction may cause thecurrent focus to retreat or backtrack through the given sequence ororder of user interface elements. Advancing or backtracking through asequence of user interface elements may include moving up or downthrough rows, screens, hierarchies, pages, and/or levels of userinterface elements.

UI 500H-UI 500I (FIGS. 5H-5I) depict an example of moving the currentfocus and the accessibility cursor using a location-dependent gesture.In this example, the accessibility cursor 502 (and current focus) isinitially on or at videos icon 145. User gesture 517, depicted in UI500H, is a stationary contact on the touch screen 112, e.g., asingle-finger tap on music application icon 146.

UI 500I (FIG. 5I) depicts that, in response to user gesture 517 (FIG.5H), the current focus, and thus, the accessibility cursor 502 moves tomusic application icon 146. Audible information associated with themusic application icon 146, such as “music” or “music application” or“tap to select music”, is also output in response to user gesture 517.Thus, the accessibility user interface may include bothlocation-dependent finger gestures for navigation that provideaccessibility information about user interface elements at the locationof the finger gesture (e.g., gesture 517), as well aslocation-independent finger gestures for navigation that provideaccessibility information about user interface elements independent ofthe location of the finger gesture (e.g., gestures 504, 506, 508, 510,and 512).

UI 500I-UI 500J (FIGS. 5I-5J) depict an example of advancing through asequence of user interface elements displayed on different userinterface screens. In UI 500I, a first screen 518 of user interfaceelements for initiating different applications on the device isdepicted, while in UI 500J, a second screen 520 of user interfaceelements is depicted. In this example, the accessibility cursor 502 (andcurrent focus) is initially on or at the music application icon 146, thelast user interface element in the first screen 518 of user interfaceelements. User gesture 519, depicted in UI 500I, has an initial contact519-1 on the touch screen 112 that moves 519-2 towards the right edge ofthe touch screen 112.

UI 500J (FIG. 5J) depicts that in response to user gesture 519 (FIG.5I), the current focus, and thus, the accessibility cursor 502 has movedfrom the first screen 518 to the first user interface element in the setof user interface elements for the second screen 520, namely, onlinevideo application icon 155. Audible information associated with theonline video application icon 155, such as “online videos” or “onlinevideos application” or “tap to select online videos”, is also output inresponse to user gesture 519. In some embodiments, the audibleinformation also includes a screen change sound to indicate that thecurrent focus has moved to a user interface element on anotherpage/screen in response to user gesture 519.

UI 500K-UI 500O (FIGS. 5K-5O) depict an example of advancing through asequence of user interface elements one character at a time.

UI 500K (FIG. 5K) depicts an exemplary accessibility user interface fortouch-based navigation among user interface elements for text entry onan electronic device in accordance with some embodiments. The currentfocus is within user name text entry element 530, where a text cursor532 is displayed preceding first text string 534 (here, the string“client”). User name text entry element 530 is for the entry of a username, and has a corresponding first user interface element label 536,“User Name.” Password text entry element 538 is for the entry of apassword, and has a corresponding second user interface element label540, “Password,” and a second text string 542, which is optionallydisplayed with replacement privacy characters.

User gesture 544, e.g., a location-independent swipe or flick gesture,has an initial contact 544-1 on the touch screen 112 that moves 544-2towards the bottom edge of the touch screen 112. Note that thedetection, interpretation, and response to gesture 544 is independent ofcontacting any particular icon displayed in the accessibility userinterface.

UI 500L (FIG. 5L) illustrates the exemplary user interface followinguser gesture 544. In response to user gesture 544, the text cursor 532has advanced one character, i.e., from the position preceding first textstring 534 within user name text entry element 530 to the positionimmediately after the “c” of the string “client.” This indicates thatthe text cursor 532 is between the “c” and the “1” of the string“client” in the first text string 534, within user name text entryelement 530. Accessibility information associated with the movement ofthe text cursor 532 such as speaking the letter “c”, is also output inresponse to user gesture 544.

UI 500L (FIG. 5L) also depicts user gesture 546 on the exemplary userinterface, e.g., another location-independent finger swipe or flickgesture. Gesture 546 has an initial contact 546-1 on the touch screen112 that moves 546-2 substantially opposite user gesture 544, i.e.,towards the top edge of the touch screen 112.

UI 500M (FIG. 5M) depicts that in response to user gesture 546, the textcursor 532 has retreated one character to return the current focus toits position preceding first text string 534 within user name text entryelement 530. Accessibility information associated with the movement ofthe text cursor 532, such as speaking the letter “c”, is also output inresponse to user gesture 546.

UI 500N (FIG. 5N) illustrates an example where the text cursor 532 is atthe end of first text string 534 within user name text entry element530. UI 500N (FIG. 5N) depicts user gesture 550 on the exemplary userinterface, e.g., another location-independent finger swipe or flickgesture. Gesture 550 has an initial contact 550-1 on the touch screen112 that moves 550-2 towards the bottom edge of the touch screen 112. Inresponse to gesture 550, the text cursor 532 remains at the end of thefirst text string 534 (FIG. 5N) and accessibility information is outputindicating that the end of the text in text entry element 530 has beenreached, such as a beep sound or other audible indicia for a border.

FIG. 5N also illustrates an alternative gesture to single-finger flickdown gesture 550, namely single-finger flick right gesture 551. Gesture551 has an initial contact 551-1 on the touch screen 112 that moves551-2 towards the right edge of the touch screen 112.

UI 500O (FIG. 5O) depicts that in response to user gesture 551 (FIG.5N), the device advances the text cursor 532 from user name text entryelement 530 (where the text cursor 532 was at the end of the first textstring 534) to the beginning of second text string 542 within passwordtext entry element 538. Accessibility information associated with themovement of the text cursor 532, such as speaking the “password” or“password field”, is also output in response to user gesture 551.

Audible Document Section Indicia

FIGS. 5P-5T illustrate exemplary accessibility user interfaces fornavigating in documents and audibly communicating document sectionindicia during touch-based document navigation in accordance with someembodiments. The user interfaces in these figures are used to illustratethe processes described below with respect to FIGS. 8A-8B.

UI 500P (FIG. 5P) depicts an exemplary user interface for communicatingdocument section indicia. Document display area 552 displays an entiredocument (if it is small enough) or a section of a document when thedocument is too large to fit within document display area 552. In UI500P, the exemplary document 553 displayed within document display area552 is too large to fit within the display area. But document 553 can bedisplayed in four sections: 553-1 (FIG. 5P), 553-2 (FIG. 5Q), 553-3(FIG. 5R), and 553-4 (FIG. 5S). In some embodiments, such as the exampledepicted here, an optional visual document section indicia 554 isdisplayed to indicate the portion of the document that is beingdisplayed and its relative location within a document. In FIG. 5P,visual document section indicia 554 is displayed near the top ofdocument display area 552 to indicate that the beginning section of thedocument 553-1 is being displayed within document display area 552.

Multi-finger user gesture 555 (FIG. 5P), which in this exemplaryembodiment is carried out with three fingers, has initial contacts555-1, 555-2, and 555-3, on the touch screen 112 that move 555-4, 555-5,and 555-6 towards the top edge of the touch screen 112. In other words,gesture 555 is a three-finger upward flick or swipe gesture (Table 1).

UI 500Q (FIG. 5Q) depicts that in response to user gesture 555, the userinterface ceases to display the beginning of the document 553-1, anddisplays the second, adjacent section of the document 553-2 withindocument display area 552. Accordingly, visual document section indicia554 is displayed in a lower portion of document display area 552.Additionally, the accessibility user interface outputs an audibledocument section indicia, such as a spoken message stating “section 2 of4”, “page 2 of 4”, or “section 2 of 4 of document being displayed.”

Multi-finger user gesture 556 (FIG. 5Q), which in this exemplaryembodiment is carried out with three fingers, has initial contacts556-1, 556-2, and 556-3, on the touch screen 112 that move 556-4, 556-5,and 556-6 towards the top edge of the touch screen 112. In other words,gesture 556 is another three-finger upward flick or swipe gesture (Table1).

UI 500R (FIG. 5R) depicts that in response to user gesture 556, the userinterface ceases to display the second section of the document 553-2,and displays a third, adjacent section of the document 553-3 withindocument display area 552. Accordingly, visual document section indicia554 is displayed in an even lower portion of document display area 552than was displayed in UI 500Q when the second section of the document553-2 was being displayed in document display area 552. Additionally,the accessibility user interface outputs an audible document sectionindicia, such as a spoken message stating “section 3 of 4”, “page 3 of4”, or “section 3 of 4 of document being displayed.”

Multi-finger user gesture 557 (FIG. 5R), which in this exemplaryembodiment is carried out with three fingers, has initial contacts557-1, 557-2, and 557-3, on the touch screen 112 that move 557-4, 557-5,and 557-6 towards the top edge of the touch screen 112. In other words,gesture 557 is another three-finger upward flick or swipe gesture (Table1).

UI 500S (FIG. 5S) depicts that in response to multi-finger user gesture557, the user interface ceases to display the third section of thedocument 553-3, and displays a fourth, adjacent section of the document553-4 within document display area 552, which in this example, includesthe end of the document. Accordingly, visual document section indicia554 is displayed at the bottom of document display area 552.Additionally, the accessibility user interface outputs an audibledocument section indicia, such as a spoken message stating “section 4 of4”, “page 4 of 4”, or “section 4 of 4 of document being displayed” or“end of document.”

Multi-finger user gesture 558 (FIG. 5S), which in this exemplaryembodiment is carried out with three fingers, has initial contacts558-1, 558-2, and 558-3, on the touch screen 112 that move 558-4, 558-5,and 558-6 towards the bottom edge of the touch screen 112 (i.e., in adirection substantially opposite user gestures 555, 556, and 557). Inother words, gesture 558 is a three-finger downward flick or swipegesture (Table 1).

UI 500T (FIG. 5T) depicts that in response to user gesture 558, the userinterface ceases to display the fourth section of the document 553-4,and redisplays the third, adjacent section of the document 553-3 withindocument display area 552. The display of visual document sectionindicia 554 is moved to the location that corresponds to displaying thethird section of the document 553-3. Additionally, the accessibilityuser interface outputs an audible document section indicia, such as aspoken message stating “section 3 of 4”, “page 3 of 4”, or “section 3 of4 of document being displayed.”

In the exemplary embodiments displayed in UI 500P-UI 500T (FIGS. 5P-5T),the sections of the document 553 are displayed as pages or documentsections arranged top-to-bottom. A three-finger flick gesture goingtowards the top of the touch screen 112 will display a document sectionthat is below and adjacent to the currently displayed document section,if another, subsequent document section can be displayed. In someembodiments (not shown), a series of pages or document sections may bearranged side-by-side for touch-based document navigation. In thoseembodiments, user gestures may be similarly oriented with theside-by-side arrangement of adjacent document sections or pages. Forexample, a three-finger user gesture going towards the right or left ofthe touch screen displays a previous or next section or page of adocument adjacent to the currently displayed section or page,respectively.

Bypass of Accessibility User Interface to Underlying User Interface

In some embodiments, a user may perform a gesture that enablespass-through of events (e.g., touch-based gestures, as well as all otheruser-inputs) to an underlying standard, non-accessibility user interfacethat processes all user inputs and applications normally (i.e., outsidethe context of the accessibility user interface). In some embodiments,the gesture used to enable pass-through is a double-tap gesture, withthe second tap remaining in contact with the touch-sensitive surface fora predefined period, e.g., a period of time between 0.05 and 1.5 seconds(Table 1).

Navigation Settings Gestures

Some accessibility navigation gestures discussed herein navigate amongtext-based user interface elements for reading or for entering text.Some gestures permit navigation among user interface elements on webpages. Still other gestures permit manipulation of controls, such asmoving settings on clocks, temperature gauges, volume settings, etc.

In some embodiments, a common gesture may be used to change settings forhow accessible navigation of text, web pages, and/or controls isperformed (e.g., in response to up or down single-finger flick or swipegestures, Table 1). For example, a two-finger rotation or twistinggesture on the touch-sensitive surface may be used to “turn” a virtualdial, rotor, or knob to choose between options (referred to hereinafteras a “rotor” or “rotor control” and gestures of this type may bereferred to herein as “rotor gestures”). This “rotor” control acts as aninvisible dial for changing the response of the device to the detectionof certain navigation gestures in the accessibility user interface.

The effect of the rotor control depends on usage context, i.e., whattype of content is currently being navigated with the accessibility userinterface. For example, when reading text in an email, the rotor mayswitch the accessible navigation from one level of navigation resolutionto another, such as changing from character-by-character navigation toword-by-word navigation. Additionally, the rotor changes the emission ofaudible text output to reflect the new accessible navigation resolution.Thus, text corresponding to the content is spoken word-by-word orcharacter-by-character in response to navigation gestures, in accordancewith the current accessible navigation resolution. As another example,the rotor may be used to alter the accessible navigation and emission ofaudible text output when browsing a webpage. For example, in response toone or more rotor gestures, the device may be set to navigate and speaktext corresponding to the web page word-by-word, navigate and speak textcorresponding to the web page character-by-character, navigate amongstweb page headers and speak text indicating what the web page's headersare, navigate amongst the links on the web page and speak those links onthe web page (all of them, just visited links, or just links not yetvisited), navigate amongst the form elements on the webpage and speakthe form elements from the web page, or navigate amongst images on theweb page and speak descriptions of those images.

Table 2 describes exemplary rotor options depending on the navigationcontext within the accessibility user interface.

TABLE 2 Exemplary navigation settings options Navigation ContextParameters/Options Changes Reading text Select and hear text bycharacter, word, sentence, line, paragraph, or page Web page browsingSelect and hear text by character, word, sentence, line, paragraph,page, headers, links, form elements, links visited, links not visited,or descriptions of images Entering text Move insertion point (textcursor) and hear text by character, by word, and/or hear suggestedauto-completion text Changing control values Select and hear value bycharacter or by word. Adjust value of control objects; date formats;time formats; language settings; units of measure; temperature scales;splitter user interface elements (i.e., a single line user interfaceelement that controls how much space a view has compared to another viewwithin an application, such as in an email application, the splitterline between the email messages table and the email message previewcontent window); horizontal, vertical, or circular sliders; or othertypes of controls that have a range of either continuous or discretevalues. User interface elements Move to user interface elements that areabove or below the current focus (i.e., the currently selected userinterface element).

In some embodiments, a rotor-based gesture includes a first fingercontact and a second finger contact that are separated by a distance,where the distance between the first finger contact and the secondfinger contact includes a center point, and the first finger contact andthe second finger contact rotate on the touch screen display around thecenter point. In essence, as discussed above, this rotation gesturesimulates turning or twisting a knob in one direction or the other. Asthe gesture progresses, the simulated knob turning changes the currentnavigable unit type.

In some embodiments, a rotor-based gesture includes placing a firstfinger contact on a touch-sensitive surface, concurrently placing asecond finger contact on the touch-sensitive surface, and rotating thesecond finger contact about the first finger contact while the firstfinger contact remains stationary. In this embodiment, the first fingercontact acts as a center point for the rotation by the second fingercontact.

For example, consider an accessible electronic device with atouch-sensitive surface where text is being read to a user one navigable“unit” at a time in response to that user's navigation requests on thetouch-sensitive surface, i.e., each of the user's navigation requestsresults in one unit being read to the user, viz. character-by-character,word-by-word, line-by-line, etc. Changing the navigable unit type isaccomplished by a navigation settings gesture, such as the rotorgesture. In some embodiments, the rotor gesture toggles the navigableunit setting between reading the text character-by-character and readingthe text word-by-word. In some embodiments, the rotor gesture changesthe navigation unit type by sequencing through unit types from smallestto largest (or vice-versa), e.g., characters are the smallest navigableunit type, then words, sentences, lines, paragraphs, and pages.

Other non-limiting examples of navigable units that may be adjusted are:web content, e.g., headers, links, visited links, etc; configurationsettings, e.g., date formats, time formats, language settings, units ofmeasure, temperature scales; splitter user interface elements (i.e., asingle line user interface element that controls how much space a viewhas compared to another view within an application, such as in an emailapplication, the splitter line between the email messages table and theemail message preview content window); horizontal, vertical, or circularsliders; or other types of controls that have a range of eithercontinuous or discrete values.

In some embodiments, the navigation settings are adjusted by a gestureperformed in a location-independent fashion on the touch-sensitivesurface of the accessible system. Thus, because the system is designedfor accessibility use, one need not contact any specific portion of thetouch sensitive surface, or any specific user interface element toadjust the current navigable unit type. In some embodiments, gesturesthat adjust the rotor are independent of contacting any user interfaceelements.

UI 500U through UI 500CC (FIGS. 5U-5CC, respectively), together with UI500K-UI 500M (FIGS. 5K-5M, respectively), are exemplary user interfacesdepicting example operations of the rotor gesture to adjust navigationsettings in the accessibility user interfaces disclosed herein.

As discussed above, UI 500K-UI 500M depict an exemplary user interfacefor touch-based navigation among user interface elements for text entryon an electronic device in accordance with some embodiments. In UI500K-UI 500M, navigation of the text cursor 532 is set on acharacter-by-character basis. For example, see the position of textcursor 532 advance and retreat one character in response to usergestures 544 (FIG. 5K) and 546 (FIG. 5L), respectively).

In UI 500U (FIG. 5U), user name text entry element 530 is selectedbecause it is highlighted by accessibility cursor 502. The current focusis within user name text entry element 530, where text cursor 532 isdisplayed preceding the text string 560 (here, the string “client li”).Rotor gesture 562 includes a first finger contact 562-1 and a secondfinger contact 562-2 that rotate or twist (562-3 and 562-4) on thetouch-sensitive surface, which simulates turning a knob or a rotor.

Though not visible in UI 500U, the electronic device changes the currentnavigable unit type from characters to words in response to detectingrotor gesture 562. Accessibility information about the navigable unittype is also be emitted in response to detecting rotor gesture 562.Here, the accessibility information indicates to the user that thecurrent navigable unit type is now set to “word” (e.g., a spoken outputis emitted saying, “words” or “navigating by words”). In this example,rotor gesture 562 is performed in a counter-clockwise rotationalpattern.

In some embodiments (not shown), the accessibility information mayinclude graphic indicia that provides a visual indication of the currentnavigable unit type.

In some embodiments, using a counter-clockwise rotational pattern willmodify current navigable unit types in increasing increments (e.g., fromcharacters, to words, to lines, to sentences, and then back tocharacters), while using a clockwise rotational pattern will modifycurrent navigable unit types in decreasing increments (e.g., fromsentences, to lines, to words, to characters, and then back tosentences). In alternative embodiments, using a counter-clockwiserotational pattern will modify current navigable unit types indecreasing increments, while using a clockwise rotational pattern willmodify current navigable unit types in increasing increments.

UI 500V (FIG. 5V) depicts user gesture 564, e.g., a finger swipe orflick gesture, on the exemplary user interface. Gesture 564 has aninitial contact 564-1 on the touch screen 112 that moves 564-2 towardsthe bottom edge of the touch screen 112.

UI 500W (FIG. 5W) illustrates that the text cursor 532 has moved oneword forward in response to gesture 564, i.e., from the beginning of thetext string “client” to the beginning of the name “li.” This movement isin contrast to the operations shown in UI 500L where, in response touser gesture 544 (FIG. 5K), the text cursor 532 advanced only onecharacter (i.e., from the position preceding first text string 534within user name text entry element 530 to the position immediatelyafter the “c” of the string “client”). The difference in operation isdue to the navigation setting gesture (rotor gesture 562) changing thecurrent navigable unit type from character to word.

In UI 500X (FIG. 5X), rotor gesture 566 includes a first finger contact566-1 and a second finger contact 566-2 that rotate or twist (566-3 and566-4) on the touch-sensitive surface, which simulates turning a knob ora rotor. The rotor gesture 566 is in a clockwise direction,substantially opposite the counterclockwise direction of rotor gesture562 in UI 500U.

Though not visible in UI 500X, the electronic device changes the currentnavigable unit type from words to characters in response to detectingrotor gesture 566. Accessibility information about the navigable unittype is also be emitted in response to detecting rotor gesture 566.Here, the accessibility information indicates to the user that thecurrent navigable unit type is now set to “character” (e.g., a spokenoutput is emitted saying, “characters” or “navigating by characters”).

UI 500Y (FIG. 5Y) depicts user gesture 568, e.g., a finger swipe orflick gesture, on the exemplary user interface. Gesture 568 has aninitial contact 568-1 on the touch screen 112 that moves 568-2 towardsthe bottom edge of the touch screen 112.

UI 500Z (FIG. 5Z) illustrates that the text cursor 532 has moved forwardone character to the position between “l” and “i” in the word “li” inresponse to gesture 568. This movement is in contrast to the operationsshown in UI 500W where, in response to user gesture 564 (FIG. 5V), thetext cursor 532 advanced one word. The difference in operation is due tothe navigation setting gesture (rotor gesture 566) changing the currentnavigable unit type from word to character.

UI 500AA (FIG. 5AA) illustrates another exemplary embodiment for usingthe rotor gesture. As noted above, rotor gestures in the accessibilityuser interface context can be used for changing the value of an optioncontrol as well as for setting navigation options for text and web pages(e.g., Table 2). In UI 500AA, language selection user interface element570 has been selected (not depicted), and in response, language selectoruser interface element 572 is displayed. Element 572 includes a numberof language options, e.g., English 572-1, Español 572-2, Français 572-3,and Italiano 572-4. Accessibility cursor 502 is at language optionEnglish 572-1.

Rotor gesture 574 includes a first finger contact 574-1 and a secondfinger contact 574-2 that rotate (574-3 and 574-4) on thetouch-sensitive surface around a center point, which simulates turning aknob or a rotor.

Though not visible in UI 500AA, the electronic device changes thelanguage option on language selector user interface element 572 inresponse to detecting rotor gesture 574. Accessibility information aboutthe currently selected language type is also be emitted in response todetecting rotor gesture 574. Here, the accessibility informationindicates to the user the current language setting (e.g., a spokenoutput is emitted saying, “English” or “Español” or “Français”).

In UI 500BB (FIG. 5BB), language option Français 572-3 on languageselector user interface element 572 has been selected as a result of oneor more rotor gestures 574. Accessibility cursor 502 is also placed atlanguage option Français 572-3.

UI 500CC (FIG. 5CC) reflects that the French language option Français572-3 on language selector user interface element 572 was selected, andthus, the language displayed on the web page has changed from English toFrench.

UI 500DD (FIG. 5DD) depicts a selection technique used in theaccessibility user interface for some embodiments (e.g., Table 1). Theaccessibility cursor 502 is on the language selection user interfaceelement 570 (here, the language is in French). Thus, alocation-independent single-finger double tap gesture 576 on thetouch-sensitive surface will activate the language selector userinterface element 572, as depicted in UI 500EE (FIG. 5EE).

In some embodiments, an alternative selection technique used in theaccessibility user interface includes depressing a physical button(e.g., Home button 204) when the accessibility cursor 502 is located ata user interface element that the user wants to activate. For example,in these embodiments, activation of Home button 204 when theaccessibility cursor 502 is at the language selection user interfaceelement 570 will activate the language selector user interface element572, as depicted in UI 500EE (FIG. 5EE).

Split Multi-finger Gestures

UI 500FF-UI 500GG (FIGS. 5FF and 5GG) illustrate that, in someembodiments, the accessibility user interface facilitates use of twofingers on one or both hands to navigate through and activate userinterface elements presented in the interface. In UI 500FF, userselection gesture 578 begins with a first point of contact 578-1 attexting application icon 141. In response, accessibility cursor 502 isplaced at texting application icon 141 and accessibility informationabout the texting application icon 141 is emitted, e.g., the spoken text“texting” or “texting application” or “tap to select texting.” As userselection gesture 578 begins to move 578-2 across the touch-sensitivesurface, the current focus moves to other user interface elements. Inthe example of UI 500FF, user selection gesture 578 moves 578-2 acrossthe stocks application icon 149-2, so the current focus is identified asbeing at the stocks application icon 149-2 as the user selection gesture578 moves over it, and accessibility information about the stocksapplication icon 149-2 is emitted.

UI 500GG depicts that the user selection gesture's point of contact hasmoved 578-3 to dictionary application icon 149-5. In response, thecurrent focus and accessibility cursor 502 are placed at dictionaryapplication icon 149-5, and accessibility information about thedictionary application icon 149-5 is emitted, e.g., the spoken text“dictionary” or “dictionary application” or “tap to select dictionary.”At this point, in response to a tap 578-4 by a second finger on thetouch-sensitive surface while the point of contact 578-3 of the firstfinger is still at a location corresponding to the dictionaryapplication icon 149-5, the dictionary application is launched.

Split multi-finger gestures in the accessibility interface are alsoeffective and applicable in many other contexts, such as: quicklynavigating over and selecting keys from a keyboard or keypad, includingphone number entry keypads; finding and activating user interfaceelements on a web page or application software program; and scrubbingaudio/video files by finding timestamp locations in the media and thentapping to activate a control. Split tapping also facilitates repetitiveactivations of the same key. For example, to delete a word, a user mayfind the delete key with one finger, and then repeatedly tap thetouch-sensitive surface with another finger to repeatedly activate thedelete button and quickly delete multiple letters.

Zoom Mode

Those with low vision, uncorrected myopia, presbyopia, or other reasonsto view a display that is magnified, may use a zoom mode. Zoomaccessibility mode permits a user to magnify the entire screen of anyapplication or user interface to assist the user to see what is on thedisplay. In some embodiments, zoom mode is not used concurrently withvoice over mode because of accessibility user interface gesture overlap.Alternatively, different gestures sets may be used with voice over modeand zoom mode so that some or all of both accessibility modes may beused concurrently.

UI 500HH (FIG. 5HH) and Table 3 describe exemplary gestures for zoommode.

TABLE 3 Exemplary zoom mode accessibility gestures Detected GestureResponse to Detected Gesture Three-finger double tap on the touch-Toggle between a zoomed-in/magnified sensitive surface. display of theuser interface elements and a normal/unmagnified display. Three-fingermovement on the touch- Pan/move the content of the displayed usersensitive surface. interface. Three-finger contact/movement reduced to aScroll/pan/move the content of the displayed one-finger contact/movement(by lifting off user interface in a direction away from the two of thefingers) in an edge or “gutter” of touched edge of the touch-sensitivesurface. the touch-sensitive surface. Three-finger double tap on thetouch- Increase zoom/magnification level of the sensitive surface, wherethe second tap displayed user interface elements. remains in contactwith the touch-sensitive surface and then moves towards the top of thetouch-sensitive surface until the desired zoom level is reached.Three-finger double tap on the touch- Decrease zoom/magnification levelof the sensitive surface, where the second tap user interface elementsdisplayed. remains in contact with the touch-sensitive surface and thenmoves towards the bottom of the touch-sensitive surface until thedesired zoom level is reached.

The accessibility gestures in Table 3 are merely exemplary. In someembodiments, the responses to opposite gestures (e.g., an upward gestureversus the corresponding downward gesture) may be reversed from thoseshown in Table 3. In some embodiments, the responses to oppositegestures are user configurable, e.g., via a settings or options menu.

With respect to the gestures and corresponding user interface actionslisted in the Table 3, UI 500II (FIG. 5II) illustrates the use of alocation-independent three-finger double tap gesture 580 (i.e., theindividual contacts 580-1, 580-2, and 580-3 represent a three-fingerdouble tap) on the touch-sensitive surface to zoom in on the userinterface so that user interface elements are magnified.

UI 500JJ (FIG. 5JJ) depicts that in response to three-finger double tapgesture 580, the accessibility user interface has magnified the userinterface so that application icons Stocks 149-2, Voice Memo 142, Alarm149-4, and Dictionary 149-5 are larger, and other portions of the userinterface are now not within the displayed portion of the userinterface.

UI 500JJ also depicts that a three-finger movement gesture 582 on thetouch-sensitive surface, which pans (i.e. moves) the content of the userinterface displayed. Specifically, three-finger movement gesture 582includes first, second, and third initial points of contact (582-1,582-2, and 582-3, respectively), followed by movement on thetouch-sensitive surface of the three points of contact (582-4, 582-5,and 582-6, respectively). Note that in this example, the three points ofcontact went downward towards the bottom of the touch-sensitive surface.

UI 500KK (FIG. 5KK) depicts that in response to three-finger movementgesture 582, the accessibility user interface has panned so that insteadof displaying user interface application icons Stocks 149-2, Voice Memo142, Alarm 149-4, and Dictionary 149-5, user interface application iconsPhotos 144 and Camera 143 are now visible in the upper portion of thedisplay, and Stocks 149-2 and Voice Memo 142, which were in the upperportion of the display, are now in the lower portion of the display.

UI 500KK also depicts use of a three-finger zoom gesture 584 to furthermagnify the user interface. Here, the gesture 584 includes athree-finger double tap on the touch-sensitive surface with threeinitial points of contact 584-1, 584-2, and 584-3. The second tapremains in contact with the touch-sensitive surface, and then moves584-4, 584-5, and 584-6, towards the top of the touch-sensitive surfaceuntil the desired zoom level is reached.

UI 500LL (FIG. 5LL) depicts that after three-finger zoom gesture 584,user interface application icon Voice Memo 142 is further magnified andnow occupies most of the display.

UI 500LL also illustrates use of a three-finger panning gesture 586 tomove around in the user interface. Specifically, three-finger panninggesture 586 includes first, second, and third initial points of contact(586-1, 586-2, and 586-3, respectively), followed by movement on thetouch-sensitive surface of the three points of contact (586-4, 586-5,and 586-6, respectively). The device moves the displayed user interfacein accordance with the movement in the gesture 586. Note that, while themovement in this example is largely in one direction (i.e., the verticaldirection), three-finger panning gestures typically result intwo-dimensional scrolling within the interface because the panninggesture movement is typically not perfectly vertical or perfectlyhorizontal on the touch sensitive surface. In some embodiments, inresponse to detecting lift off of the three-finger panning gesture, thedevice continues to pan the displayed user interface with deceleratingvelocity in accordance with the speed of the fingers at or near lift offin the three-finger panning gesture.

UI 500MM (FIG. 5MM) depicts that in response to three-finger panninggesture 586 (FIG. 5LL), the dictionary application icon 149-5 nowoccupies most of the display.

UI 500MM also depicts use of a three-finger zoom out gesture 588 todemagnify the user interface. Gesture 588 includes a three-finger doubletap on the touch-sensitive surface with three initial points of contact588-1, 588-2, and 588-3. The second tap remains in contact with thetouch-sensitive surface, and then moves 588-4, 588-5, and 588-6 towardsthe bottom of the touch-sensitive surface until the desired zoom levelis reached.

UI 500NN (FIG. 5NN) depicts that in this example, the magnificationlevel selected by three-finger zoom out gesture 588 returns the displayto a 1.0× magnification factor, so that the user interface display is inthe same magnification/zoom state as it was before using the zoom mode.

In some embodiments, after the recognition of a three-finger panninggesture (e.g., 586, FIG. 5 l), a forgiveness technique is used thatallows smooth panning as up to two of the three fingers at a time losecontact with the touch-sensitive surface. The forgiveness techniqueincludes the ability to lose and regain contact with any given fingerduring the panning gesture, while still providing smooth panning so longas one of the three fingers regains contact with the touch-sensitivesurface within a short, predefined time period. In essence, so long as auser starts a panning gesture with three fingers, the user can pan withjust one finger, as well as change which finger is being used to directthe pan. This technique is beneficial because users rarely perform athree-finger gesture by causing all three fingers to make contact with atouch-sensitive surface simultaneously in a single combined event.Similarly, users seldom have all three fingers leave a touch-sensitivesurface at the same time. Accordingly, these techniques determine that athree-finger panning gesture is occurring, and gracefully handle when auser is still performing the three-finger panning gesture, but the userdoes not keep all three fingers in contact with the touch-sensitivesurface.

In some embodiments, an electronic device with a display and atouch-sensitive surface executes program instructions that store fingertouch events on the touch-sensitive surface to detect when athree-finger gesture happens. If fewer than three points of contact of athree-finger gesture are detected, whatever touch events have happenedare processed in the order they were received. In response to detectingat least three concurrent initial points of contact on thetouch-sensitive surface, the device assigns one of the three initialpoints of contact on the display to be a first primary contact. Thedevice then determines whether the first primary contact remains incontact with the touch-sensitive surface, and the device then detectsmovement of the first primary contact. In response to detecting movementof the first primary contact, the device pans a plurality of userinterface elements on the display in accordance with the movement of thefirst primary contact. Then, if and when the device detects the firstprimary contact losing contact with the touch-sensitive surface, thedevice assigns a second primary contact selected from the at least threepoints of contact on the touch-sensitive surface, wherein the secondprimary contact is still in active contact with the touch-sensitivesurface (or the second primary contact is once again in active contactwith the touch-sensitive surface). Further, to facilitate smoothpanning, the device calculates an offset between the first primarycontact and the second primary contact. At that point, once the devicedetects movement of the second primary contact, the device pans theplurality of user interface elements on the display in accordance withthe movement of the second primary contact and the offset in response todetecting movement of the second primary contact.

The sequence of reassigning a primary contact when the old primarycontact is lost, calculating an offset between the old primary contactand the reassigned primary contact, and moving the user interface on thedisplay in accordance with the movement of the new, reassigned primarycontact and the calculated offset is repeated for as many times as theuser chooses to keep panning, so long as either: (1) there is always atleast one finger in contact with the touch-sensitive surface; or (2)there is no more than a predefined delay between reestablishing at leastone contact on the touch-sensitive surface and the time when the last ofthe three points of contact were removed from the touch-sensitivesurface. The predefined delay may be less than a second in someembodiments; in alternative embodiments, the predefined delay may bebetween 0 and 0.35 seconds, or the predefined delay may be between 0.01seconds and 0.55 seconds.

It will be evident to those with skill in the art that though threefinger contacts are discussed, this technique is also readily applicableto any finger panning or dragging gesture that uses at least twofingers. Further, a change from an old primary contact to a new primarycontact may involve any finger contact, not just one selected from theoriginal finger contacts.

In some embodiments, the accessibility user interface provides anability to magnify about any point on the touch screen display inresponse to a multi-finger gesture, including points at the edges orcorners of the display.

UI 500OO-UI 500PP (FIGS. 5OO-5PP) illustrate a multi-finger gesture 591with three finger contacts 591-1 a, 591-1 b, and 591-1 c at variouslocations on the touch screen 112. Accordingly, a bounding box 592 forthe concurrent multi-finger gesture 591 is determined.

The accessibility user interface magnifies the content (e.g., graphics)in the user interface about a point 593 in the bounding box to a secondmagnification, greater than the first magnification, on the touch screen112. The point in the bounding box is at a default position (e.g., acentroid) within the bounding box when the bounding box is located atthe center of the touch screen display. For example, the point 593-1 ais at the center of the bounding box because the bounding box 592-1 isat or nearly at the center of touch screen 112.

As shown in other exemplary bounding boxes in UI 500OO, placement of themulti-finger gesture in different locations on the touch screen 112results in the determination of bounding boxes 592 in correspondinglocations, and each corresponding point 593 in the bounding box isdisplaced from a default position of the bounding box towards arespective edge of the bounding box by an amount that increases as adistance that the bounding box is displaced from the center of the touchscreen display towards a corresponding edge of the touch screen displayincreases. For example, bounding box 592-2 is closer to the right edgeof touch screen 112, so point 593-1 b is closer to the right edge of thebounding box. Similarly, bounding box 592-4 is closer to the bottom edgeof touch screen 112, so point 593-1 d is closer to the bottom edge ofthe bounding box.

Bounding boxes 592-3 and 592-5 are located at the right and bottom edgesof the touch screen 112, respectively, and thus, points 593-1 c and593-1 e are displaced to the right and bottom edges of bounding boxes592-3 and 592-5, respectively.

Accordingly, in some embodiments, the accessibility user interface candisplace a point in a bounding box towards the respective edge of thatbounding box by an amount that is proportional to the distance that thebounding box is displaced from the center of the touch screen displaytowards the corresponding edge of the touch screen display. Further, theaccessibility user interface can displace points within bounding boxesin two directional axes, rather than just one axis.

UI 500PP illustrates exemplary bounding boxes that displace pointswithin bounding boxes in two directional axes. Exemplary bounding box592-6 has a point 593-1 f that is not displaced from the center of thebounding box due to the placement of the bounding box at the center oftouch screen 112. Bounding box 592-7, however, has point 593-1 g, whichis displaced from the default position (e.g., the centroid) of boundingbox 592-7. Bounding box 592-7 is located closer to the upper-rightcorner of touch screen 112. Accordingly, the point 593-1 g withinbounding box 592-7 may be displaced from the default position of thebounding box 592-7 towards the right and top edges of the bounding box592-7 by an amount that is proportional to the distance that thebounding box is displaced from the center of the touch screen displaytowards the right and top edges of the touch screen 112.

Bounding box 592-8 has a point 593-1 h that is displaced from thecentroid of bounding box 592-8. Bounding box 592-8 is placed at theupper-right corner of touch screen 112. Thus, the point 593-1 h withinbounding box 592-8 is displaced from the default position of thebounding box 592-8 to the right and top edges of the bounding box 592-8.Thus, the accessibility user interface provides an ability to magnifyabout any point on the touch screen display in response to amulti-finger gesture, including points at the edges or corners of thedisplay.

In some embodiments, the accessibility user interface provides anability to scroll using both multi-finger gestures and single fingergestures. Referring to UI 500KK (FIG. 5KK), the figure illustrates thatin response to three-finger movement gesture 582 (FIG. 5JJ), theaccessibility user interface scrolls so that instead of displaying userinterface application icons Stocks 149-2, Voice Memo 142, Alarm 149-4,and Dictionary 149-5, user interface application icons Photos 144 andCamera 143 are now visible in the upper portion of the display, andStocks 149-2 and Voice Memo 142 (which were in the upper portion of thedisplay) are now in the lower portion of the display.

UI 500QQ (FIG. 5QQ) depicts that in some embodiments, after the devicedetects lift off of all but one of the plurality of concurrent fingercontacts, the user interface will cease scrolling of the content in theuser interface when the single remaining finger contact 597 is locatedin a predefined central portion 598 of the touch screen 112 (e.g.contact 597-1, FIG. 5QQ). Predefined border portion 599 is the area oftouch screen 112 outside of predefined central portion 598. Forillustrative purposes, predefined border portion 599 is depicted withhash marks.

UI 500QQ also depicts exemplary single remaining finger contact 597 intwo other locations. Single remaining finger contact 597-2 is locatednear the interior edge 599-1 of predefined border portion 599, whichresults in scrolling of the content in the user interface away from theedge of the touch screen that is nearest to single remaining fingercontact 597-2 (scrolling not depicted in the figures). To wit, if thesingle remaining finger contact is nearest to the left edge of thescreen, the content in the user interface will scroll toward the rightedge of the screen, revealing user interface content that was beyond theleft edge of the touch screen display's edge.

Single remaining finger contact 597-3 is located near the exterior edge599-2 of predefined border portion 599, which will also result inscrolling of the content in the user interface away from the edge of thetouch screen that is nearest to single remaining finger contact 597-3(scrolling not depicted in the figures).

In some embodiments, the scrolling speed used when detecting a singleremaining finger contact such as 597-2 or 597-3 within the predefinedborder portion 599 inversely corresponds to the distance between thesingle remaining finger contact and the edge of the touch screen 112that is nearest to the single remaining finger contact. Thus, singleremaining finger contact 597-2 would scroll the content in the userinterface slower than single remaining finger contact 597-3 would scrollthe content in the user interface.

Thus, the device may use multi-finger gestures with the same number ofconcurrent fingers (e.g., three fingers in Table 3) to initiatezoom-related actions, but also allows single-finger gestures (whichobscure less of the display) to be used seamlessly with the multi-fingergestures in zoom mode,

Mapping User Interface Containers to a Touch-Sensitive Surface

FIGS. 6A-6T illustrate accessibility user interfaces that map respectiveuser interface containers to a touch-sensitive surface in accordancewith some embodiments. The container mappings are part of an efficientaccessibility method for navigation through visually complex displays(e.g., multi-window displays). Because the mapped touch-sensitivesurface represents only a currently selected container and not theentire display, a user cannot stray into other areas of the display byaccident.

As illustrated below, a user interface container is a discrete,predefined region of a user interface on a display. User interfacecontainers contain user interface elements. Containers are typicallydefined by an application developer at the time an application isdesigned. To use containers in an accessibility user interface, adeveloper creates groups of user interface containers by organizingcertain graphical views together in an overall view. The application andthe accessibility user interface are used together to allow a user tonavigate within groups of user interface containers in a logical,understandable fashion, even when a user may not be able to visually seethe arrangement of containers. User interface containers may beorganized in a predefined, hierarchical fashion, as will be discussedbelow in greater detail.

In some embodiments, the accessibility user interfaces with containersoperate on an electronic device with a display and a touch-sensitivesurface (e.g., a track pad), such as a desktop computer or a laptopcomputer. In some embodiments, the accessibility user interfaces withcontainers operate on an electronic device with a touch screen display.In some embodiments, the touch screen display may be used with one ormore additional touch-sensitive surfaces, or one or more additionaldisplays. In some embodiments, the accessibility user interfaces withcontainers are used on portable multifunction devices (e.g., 100, FIG.1A) or other electronic devices (e.g., 300, FIG. 3).

In FIGS. 6A-6T, gestures are depicted on a touch-sensitive surface 603(e.g., a track pad) that is separate from the display. That theexemplary gestures are illustrated in these figures on a separatetouch-sensitive surface 603 does not require that the gestures beperformed on a track pad (or other separate touch-sensitive surface) topractice the methods disclosed herein. For example, in some embodiments,the gestures may be performed on a touch screen instead. For theseembodiments, a selected container may be tactilely mapped to the entiretouch screen surface while the touch screen continues to displaymultiple containers, including the selected container. The accessibilityuser interfaces in FIGS. 6A-6T are used to illustrate the processesdescribed below with respect to FIGS. 10A-10C.

Table 4 describes exemplary accessibility gestures for containernavigation and activation. These gestures are typically finger gestures.In some embodiments, the accessibility gestures for containers may alsoinclude other inputs (e.g., keyboard inputs such as holding down amodifier key, or mouse inputs) in conjunction with or in place of thefinger gestures.

TABLE 4 Exemplary accessibility gestures for container navigation andactivation Navigation and Activation Detected Gesture Response toDetected Gesture Two-finger swipe right on the touch- Select a nextlower-level container sensitive surface. associated with where thecurrent focus is set. Two-finger swipe left on the touch- Select a nexthigher-level container above sensitive surface. where the current focusis set, until the current focus is at the highest level. Single-fingermoving/dragging on the touch- Move the current focus in the currentsensitive surface. container in accordance with the mapping of thecurrent container on the touch-sensitive surface and the currentposition of the single finger on the touch-sensitive surface. Hold downa key (e.g., a shift key) while Move the current focus along ahorizontal or performing a single-finger moving vertical axis within thecurrent container as navigation gesture. the single finger moves on thetouch- sensitive surface. Two-finger double tap. Activate designatedaccessible operation associated with the region on the touch- sensitivesurface where the two-finger double tap occurred (e.g., activating theapplication chooser depicted in UI 600L). Zig-zag motion on thetouch-sensitive Exit a modal state in the current application surface(i.e., moving a single finger left to or user interface (e.g., exit amenu that is right to left, or right to left to right, without open;exit an alert displayed by the computer lifting the finger contact fromthe touch- system; exit a dialog window). sensitive surface during thezig-zag motion).

In addition, navigational gestures discussed above with respect to Table1 and Table 2 (e.g., flick navigation, rotor control for settings, etc.)are also applicable to container navigation and activation.

The accessibility gestures in Table 4 are merely exemplary. In someembodiments, a two-finger depinch gesture (instead of a two-finger swiperight gesture) is used to select a next-lower-level container belowwhere the current focus is set. In some embodiments, a two-finger pinchgesture (instead of a two-finger swipe left gesture) is used to select anext-higher-level container above where the current focus is set. Insome embodiments, the responses to opposite gestures (e.g., a rightwardgesture versus the corresponding leftward gesture, or an upward gestureversus the corresponding downward gesture) may be reversed from thoseshown in Table 4. For example, a two-finger swipe right gesture may beused to select a next-higher-level container and a two-finger swipe leftgesture may be used to select a next-lower-level container. In someembodiments, the responses to opposite gestures are user configurable,e.g., via a settings or options menu.

FIG. 6A depicts three exemplary mappings of user interface containers totouch-sensitive surface 603. In some embodiments, touch-sensitivesurface 603 is touch pad 355 on device 300 (FIG. 3).

In the first example (FIG. 6A-1), user interface container 604 isproportionally mapped to touch-sensitive surface 603. User interfacecontainer 604 includes a plurality of user interface elements, e.g.,604-1, 604-2, etc. Exemplary user interface elements within userinterface container 604 may be include, without limitation, applicationicons, web links, file names, file listings, file icons, graphical userinterface controls, menu bars, spreadsheets, presentation files orslides, lists, headings, text fields, text areas, password fields,sliders, splitters, disclosure triangles, and/or tables.

In the second example (FIG. 6A-2), a much shorter, but wider userinterface container 608 for a menu bar is proportionally mapped totouch-sensitive surface 603. Container 608 contains user interfaceelements for a drop-down menu bar, i.e., “File,” “Edit,” etc. Theproportional mapping in FIG. 6A-2 vertically stretches these userinterface elements on the touch-sensitive surface 603. This verticalstretching makes it easier for a visually impaired user to touch alocation on the touch-sensitive surface 603 that corresponds to a givenuser interface element.

In the third example (FIG. 6A-3), application icon interface container610 is proportionally mapped to touch-sensitive surface 603. Applicationicon interface container 610 contains application icon user interfaceelements Phone 138, Mail 140, Browser 147, and Music 146. Like FIG.6A-2, the proportional mapping in FIG. 6A-3 vertically stretches theseuser interface elements on the touch-sensitive surface 603. Thisvertical stretching makes it easier for a visually impaired user totouch a location on the touch-sensitive surface 603 that corresponds toa given user interface element.

UI 600B-UI 600G (FIGS. 6B-6G) depict exemplary accessibility userinterfaces for navigating in a hierarchical (nested) arrangement of userinterface containers.

Referring to UI 600B (FIG. 6B), window 612 is an overall user interfaceelement, here, a window entitled “Window.” Current container indicator614-1 (e.g., a bold or otherwise visually highlighted border) is used toidentify that the overall user interface element window 612 is presentlyselected. In some embodiments, as depicted here, current containerindicator 614 may border a title of a user interface element such as awindow. In some embodiments, a current container indicator borders theentire user interface element that is the current container. If thecurrent container is a window, the current container indicator mayborder the entire window.

In this example, window 612 is proportionally mapped 615 totouch-sensitive surface 603 because the current container is the overalluser interface element window 612.

User interface container selection event 616, in this example, atwo-finger swipe gesture, has initial points of contact 616-1 and 616-2,followed by movement (616-3 and 616-4) of these initial points ofcontact. Note that user interface container selection event 616 islocation independent. Event 616 occurs at an arbitrary angle andlocation on the touch-sensitive surface 603. In contrast, gestures fornavigating and activating within a given container (e.g., single-fingermoving/dragging gestures, Table 4) are location-dependent gestures onthe mapped touch-sensitive surface.

UI 600C (FIG. 6C) illustrates that in response to detecting userinterface container selection event 616, current container indicator614-2 has changed position to an interior portion of window 612 toreflect that the portion of window 612 presently outlined by currentcontainer indicator 614-2 is the container that is currently selected.In response to user interface container selection event 616, theinterior portion of window 612 outlined by current container indicator614-2 is proportionally mapped 620 to touch-sensitive surface 603. Thedevice may also output accessibility information associated with thecurrently selected container in response to detecting the user interfacecontainer selection event 616, such as speaking “interacting withcontainer, three items.”

In this example, the current container (the area of window 612 withincurrent container indicator 614-2) is nested within the precedingcontainer (window 612). The two containers form a hierarchy with window612 at the outermost level depicted here, and the area of window 612within current container indicator 614-2 at the next lower level in thehierarchy.

UI 600C also shows an exemplary user interface container selection event618, in this example, another two-finger swipe gesture ontouch-sensitive surface 603, that has initial points of contact 618-1and 618-2, followed by movement (618-3 and 618-4) of these initialpoints of contact. Note that the orientation of user interface containerselection event 618 is at a different angle than user interfacecontainer selection event 616. Interface container selection event 618is also location independent.

UI 600D (FIG. 6D) illustrates that in response to detecting userinterface container selection event 618, current container indicator614-3 has changed position to a new interior portion of window 612 toreflect that the portion of window 612 outlined by current containerindicator 614-3 is the container that is currently selected. Thus, insome embodiments, orientation of user interface container selectionevents is also irrelevant, as demonstrated through the orientation ofuser interface container selection event 618 which is at a differentangle than user interface container selection event 616.

In response to user interface container selection event 618, theinterior portion of window 612 outlined by current container indicator614-3 is proportionally mapped 624 to touch-sensitive surface 603. Thedevice may also output accessibility information associated with thecurrently selected container in response to detecting the user interfacecontainer selection event 618, such as speaking “interacting withcontainer, two items.”

In this example, the containers corresponding to indicators 614-1,614-2, and 614-3 (i.e., window 612, the area of window 612 withincurrent container indicator 614-2, and the area of window 612 withincurrent container indicator 614-3) form a hierarchy of nestedcontainers. Window 612 is at a first level, the area of window 612depicted with current container indicator 614-2 in UI 600C is at asecond, lower level, and the area of window 612 within current containerindicator 614-3 in UI 600D is at a third, still lower level in thehierarchy.

UI 600D also shows an exemplary user interface container selection event622, in this example, another two-finger swipe gesture ontouch-sensitive surface 603, that has initial points of contact 622-1and 622-2, followed by movement (622-3 and 622-4) of these initialpoints of contact.

UI 600E illustrates another level in the hierarchy of user interfacecontainers within window 612. In response to user interface containerselection event 622, current container indicator 614-4 has again changedposition to a new interior portion of window 612 to reflect that theportion of window 612 outlined by current container indicator 614-4 isthe container that is currently selected. In response to user interfacecontainer selection event 622, the interior portion of window 612outlined by current container indicator 614-4 is proportionally mapped625 to touch-sensitive surface 603. The device may also outputaccessibility information associated with the currently selectedcontainer in response to detecting the user interface containerselection event 622, such as speaking “interacting with text area.”

UI 600E also includes an exemplary user interface navigation gesture629, in this example, a single-finger moving on the touch-sensitivesurface 603, that has initial point of contact 629-1, followed bymovement 629-2 of the initial point of contact.

UI 600F (FIG. 6F) illustrates movement of the current focus 630 withinthe current container in accordance with the user interface navigationgesture 629. Here, the current container is a text field user interfaceelement, namely the text box that begins “Apple leads the way . . . .”The text field user interface element acts as a container because thetext/words within the text field act as “sub-elements” that may benavigated using a finger gesture on the mapped touch-sensitive surface603. For a text field user interface element, the current focus 630 mayreside at a sub-element within the text field (e.g., a word or acharacter, depending on the current setting of the rotor]). In UI 600F,the position of the current focus 630 corresponds to the location of themoving single finger contact 629-3 on the mapped touch-sensitive surface603. The device may also output accessibility information associatedwith the current focus 630 in response to detecting the user interfacenavigation gesture 629, such as speaking/outputting the word at thecurrent focus (i.e., “leads”).

Thus, the accessibility user interface may include bothlocation-independent finger gestures (e.g., 616, 618, 622, and 628) fornavigating between nested containers and location-dependent fingergestures (e.g., 629) for navigating within a current container, wherethe current container is mapped to the touch-sensitive surface. Bothtypes of gestures typically also output accessibility information aboutthe navigation, such as audible information.

UI 600F also includes an exemplary user interface container deselectionevent 628, in this example, a two-finger swipe gesture that has initialpoints of contact 628-1 and 628-2, followed by movement (628-3 and628-4) of these initial points of contact to the left of thetouch-sensitive surface.

UI 600G (FIG. 6G) illustrates that after user interface containerdeselection event 628, the current container indicator 614 has traversedback up the hierarchy of nested containers to the previous containerdefined by current container indicator 614-3.

UI 600H-UI 600J (FIGS. 6H-6J) illustrate a hierarchy of user interfacecontainers for accessible navigation in a music and video playerapplication, such as Apple Corporation's iTunes® application.

In UI 600H (FIG. 6H), the main application window 634 is the overalluser interface element. Current container indicator 636-1 is used toidentify that the overall user interface window 634 is the presentlyselected container. A user interface container selection event 638, inthis example, a two-finger swipe gesture on the touch-sensitive surface603, has initial points of contact 638-1 and 638-2, followed by movement(638-3 and 638-4) of these initial points of contact to the right.

UI 600I (FIG. 6I) illustrates that in response to detecting userinterface container selection event 638, current container indicator636-2 has changed position to a new interior portion of applicationwindow 634 to reflect that the portion of window 634 presently outlinedby current container indicator 636-2 is the container that is currentlyselected. In response to user interface container selection event 638,the interior portion of window 634 outlined by current containerindicator 636-2 is proportionally mapped to touch-sensitive surface 603.The device may also output accessibility information associated with thecurrently selected container in response to detecting the user interfacecontainer selection event 638, such as speaking “podcast, table.”

UI 600I also includes an exemplary user interface navigation gesture640, in this example, a single-finger moving on the touch-sensitivesurface 603, that has initial point of contact 640-1, followed bymovement 640-2 of the initial point of contact.

UI 600J (FIG. 6J) illustrates movement of the current focus 641 withinthe current container in accordance with the user interface navigationgesture 640. In UI 600J, the position of the current focus 641corresponds to the location of the moving single finger contact 640-3 onthe mapped touch-sensitive surface 603. The device may also outputaccessibility information associated with the current focus 641 inresponse to detecting the user interface navigation gesture 640, such asspeaking the words at the current focus (“temp puts . . . selectedrow”).

Thus, music and video player applications, as well as many othersoftware applications, may be navigated as a series of user interfacecontainers, where the current container is mapped to the touch-sensitivesurface.

UI 600K-UI 600S (FIGS. 6K-6S) illustrate exemplary accessibility userinterfaces for navigation.

User interface 644 (UI 600K) includes numerous user interface elements,including a desktop 646, a menu bar 650, and a dock 654. When theaccessibility user interface is active, current container indicator 648is visible at or around at least one of the user interface elements. InUI 600K, the current container indicator 648-1 is at menu bar 650, whichincludes different drop down menu titles, i.e., File 650-1, Edit 650-2,View 650-3, Go 650-4, Tools 650-5, etc. Menu bar 650 is mapped to thetouch-sensitive surface 603 (e.g., analogous to the menu bar mappingshown in FIG. 6A-2). The menu bar item Tools 650-5 is highlightedbecause it is the current focus within the current container (menu bar650).

UI 600K also illustrates a user interface container selection event 656,which in this example, is a two-finger split tap at locations 656-1(which corresponds to Tools 650-5) and 656-2. Any suitable gesture maybe used for navigating between containers, including without limitation,two-finger swipe right/left gestures, or pinch and depinch gestures.Container selection events may include location-dependent gestures(e.g., split taps) and location-independent gestures (e.g., two-fingerswipe gestures).

UI 600L (FIG. 6L) illustrates that in response to user interfacecontainer selection event 656, the tools menu becomes the currentcontainer, and is therefore highlighted by current container indicator648-2. The area of the tools menu is mapped to touch-sensitive surface603 (not shown) and accessibility information is emitted (e.g., speakingthe words “tools menu”). The accessibility user interface may also showa finger location indicator 659 within the current container, whichshows a position in the container that corresponds to the currentposition of a finger on the mapped touch-sensitive surface 603 (notshown). In UI 600L, finger location indicator 659 is over menu itemSelect Tool 658, which has the current focus and is thereforehighlighted. Accessibility information is emitted as the current focusmoves in accordance with the finger on the mapped touch-sensitivesurface 603 (e.g., speaking the words “tools, selected”).

In the example of UI 600L, at some time after the current container wasset to the tools menu, a user initiated an accessibility feature forchoosing applications. Any suitable method for initiating an applicationchooser may be used, such as a two-finger double-tap gesture in area 688of the touch-sensitive surface 603 (FIG. 6T) or key combinations on akeyboard or Braille device.

In this example, in response to detecting a request to start theapplication chooser, the accessibility user interface UI 600L displaysan application chooser dialog box 660 with a list of applications (e.g.,currently running applications). In some embodiments, the dialog box ismapped to the touch-sensitive surface for application selection. In someembodiments, a dialog box is not displayed. Instead, the possibleapplication choices are audibly emitted. In UI 600L, the applicationchoices include Preview 660-1 (the application with the current focus),Mail 660-2, and Dock 660-3.

In this example, the user navigates within the application chooserdialog box's options to Dock 660-3 and selects that application toswitch to (navigation and selection are not depicted in the FIG. 6L).Navigation within an application chooser dialog box 660 may beaccomplished by any suitable method, including without limitation keypresses, finger gestures, mouse cursor movement, etc.

UI 600M (FIG. 6M) depicts that in response to the user selecting theDock 660-3, the current container indicator 648-3 highlights the dockarea 654 at the bottom of the user interface. At some time afterchoosing the Dock 660-3, the user again initiates the applicationchooser, which results in the application chooser dialog box 662 beingdisplayed. Here, the application choices include Dock 662-1 (which ismarked as current because it is the application with the current focus),Mail 662-2, and Finder 662-3.

Continuing this example, the user navigates within the applicationchooser dialog box's options to Mail 662-2 and selects that applicationto switch to (navigation and selection within the application chooserdialog box are not depicted in FIG. 6M).

UI 600N (FIG. 6N) depicts that in response to the user selecting theMail 662-2 application, the accessibility user interface switches to themail application 664. Mail application 664 includes an email list area664-1 (which is the current container as indicated by current containerindicator 648-4) as well as preview area 664-2. In some embodiments, asdepicted here, an area that is not the current container will bedisplayed in a dimmed manner or with a light colored overlay, while thearea that is the current container will be displayed normally.

UI 600O (FIG. 6O) illustrates that the user has initiated a windowchooser function that is available in some embodiments (e.g., inresponse to a two-finger double-tap gesture in area 684 of thetouch-sensitive surface 603, FIG. 6T). As with the application chooser,in some embodiments, a dialog box with the window chooser options isdisplayed. In some embodiments, the dialog box is mapped to thetouch-sensitive surface for window selection. Here, window chooserdialog box 666 shows possible selections Current 666-1 and Selected item666-2. Current 666-1 refers to the window or portion of an applicationthat is the current container, here, email list area 664-1. Selecteditem 666-2 in this example is the window that includes the preview area664-2 for the email currently selected in the email list area 664-1(e.g., “Absolutely Free Image Hosting”).

In this example, the user navigated within the window chooser dialogbox's options to Selected item 666-2 and selected that window to switchto (navigation and selection within the window chooser dialog box 666are not depicted in FIG. 6O). Navigation within an window chooser dialogbox 666 may be accomplished by any suitable method, including withoutlimitation key presses, finger gestures, mouse cursor movement, etc.

UI 600P (FIG. 6P) illustrates that in response to the user's selectionof the Selected item 666-2 in the window chooser dialog box 666, thepreview area 664-2 is highlighted with current container indicator 648-5to indicate it is the current container.

Preview area 664-2 displays email 670, which has lines 670-1 through670-6. The current focus within preview area 664-2 is on line 670-3. Afinger location indicator 672-1 (analogous to indicator 659 in UI 600L)is on line 670-3, which has the current focus and is thereforehighlighted. Accessibility information is emitted as the current focusmoves in accordance with the finger on the mapped touch-sensitivesurface 603 (e.g., speaking the words “Simple, fast, absolutely freeimage hosting and sharing service for you”).

Navigation Along a Confined Axis within a Container

UI 600P-UI 600S (FIGS. 6P-6S) illustrate navigation within a userinterface container where the navigation is confined to an axis (e.g., ahorizontal axis within the container). Such navigation may help a userto select and hear words along one line on a page that includes multiplelines of words, or traverse a row or column in a 2-dimensional array ofuser interface elements (e.g., a spreadsheet or an array of icons).Navigation along a confined axis improves the speed and efficiency withwhich a user can navigate within a container because the movement of theuser's finger on the mapped touch-sensitive surface does not need to beas precise as for normal navigation on the touch-sensitive surface.

In some embodiments, navigation along a confined axis within a containeroccurs when a user adds a modifying input to a finger gesture (e.g.,holding down a keyboard key (e.g., a shift key) while performing afinger navigation gesture on the touch-sensitive surface) so that anentire line is easily read even if the finger navigation gesture doesnot stay strictly aligned with the line being read.

In UIs 600P-600S, lines 670-1 through 670-6 each have an axis 671 alongwhich the user interface elements, i.e., words, are aligned. To navigatealong one line of the plurality of user interface elements, a userinterface navigation gesture may include moving the current focus of thenavigation gesture in accordance with a gesture movement, whereinlocations for the current focus are substantially confined to the axisof the line being navigated.

For example, in UI 600Q, finger navigation gesture 674 has an initialcontact 674-1, and a movement 674-2 that is not exactly horizontal. InUI 600R (FIG. 6R), the current focus 672-2 has moved along the axis671-3, which is aligned with line 670-3, in accordance with thehorizontal component of movement 674-2, even though the movement 674-2as a whole is not exactly horizontal.

But, if the finger movement substantially departs from the axis, thecurrent focus changes to a user interface element away from the axis ofthe line that was being navigated. For example, in UI 600R, userinterface navigation gesture 676 has an initial contact 676-1, and amovement 676-2 in a direction that substantially departs from thehorizontal direction. Accordingly, in UI 600S (FIG. 6S), the currentfocus 672-3 has moved vertically to the line 670-4, rather thancontinuing to move horizontally along the axis 671-3.

Regional Touch Surface Allocation for Particular Navigation Gestures

In some embodiments, predefined areas of the touch-sensitive surface 603(e.g., exemplary areas 682, 684, 686, and 688 in FIG. 6T) may bedesignated for particular accessible operations when a particular typeof gesture is detected (e.g., a two-finger double tap gesture). Area 682is designated for making the current container the menu bar container650 (e.g., when a two-finger double tap gesture is detected in area682). Area 684 is designated for activating the window chooser 666(e.g., when a two-finger double tap gesture is detected in area 684).Area 686 is designated for making the current container the dock 654(e.g., when a two-finger double tap gesture is detected in area 686).Area 688 is designated for activating the application chooser 660 (e.g.,when a two-finger double tap gesture is detected in area 688). For thistype of gesture, the response depends on the location of the gesture onthe touch-sensitive surface, but the response is independent of themapping of the current container to the touch-sensitive surface.

FIGS. 7A-7C are flow diagrams illustrating an accessibility method fortouch-based navigation among user interface elements in accordance withsome embodiments. The method 700 is performed at an electronic devicesuch as a multifunction device (e.g., 300, FIG. 3) with a display and atouch-sensitive surface in conjunction with accessibility module 129.Some operations in method 700 may be combined and/or the order of someoperations may be changed.

In some embodiments, the touch-sensitive surface is separate from thedisplay. In some embodiments, the touch-sensitive surface is a touch pad(e.g., 355, FIG. 3).

In some embodiments, the display is a touch screen display (e.g., 112,FIG. 2) and the touch-sensitive surface is on the display (750). In someembodiments, the method is performed at a portable multifunction devicewith a touch screen display (e.g., portable multifunction device 100).

As described below, the method 700 provides an efficient way to uselocation-independent gestures to navigate though user interface elementsand provide accessibility information about the elements. The methodreduces the cognitive burden on a user with impaired vision when usingan electronic device with a display and a touch-sensitive surface,thereby creating a more efficient human-machine interface. Forbattery-operated devices, enabling a user to navigate faster and moreefficiently conserves power and increases the time between batterycharges.

The device displays (702) a plurality of user interface elements on thedisplay (e.g., UI 500A, FIG. 5A and first screen 518 of user interfaceelements in FIG. 5I). A current focus is on a first user interfaceelement. As noted above, when executing accessibility methods, a currentfocus may be established at a given user interface element in the userinterface (e.g., at texting icon 141 where accessibility cursor 502 isdisplayed, FIG. 5A). Accordingly, when the current focus is on a givenuser interface element (e.g., a window, menu item, dialog box, button,positional control, or other graphical user interface element), thecurrently selected user interface element is activated or deactivatedupon detecting appropriate user input. For example, referring to FIG.5A, if the texting icon 141 has the current focus, the texting module141 will be activated in response to detecting user input for activatingthe user interface element with the current focus (e.g., a single-fingerdouble tap gesture).

The device detects (704) a first finger gesture (e.g., a swipe gesture504 (FIG. 5A) as discussed above; or flick right, Table 1) on thetouch-sensitive surface. The first finger gesture is independent ofcontacting a location on the touch-sensitive surface that corresponds toa second user interface element. In method 700, it is immaterial whetherthe first finger gesture does or does not contact a location on thetouch-sensitive surface that corresponds to any of the plurality of userinterface elements, including the user interface element being navigatedfrom and the user interface element being navigated to in response tothe first finger gesture. In this situation, the location of the firstfinger gesture on the touch-sensitive surface does not influence themethod; the method simply requires that the first finger gesture isdetected, and that gesture may contact a location on the touch-sensitivesurface that corresponds to zero or more user interface elements fromthe plurality of user interface elements, including the second userinterface element, without deviating from the method's teachings. Thus,in method 700, gesture-based navigation from one user interface elementin a plurality of user interface elements to another user interfaceelement in that plurality does not require detecting that the firstfinger gesture's point of contact on the touch-sensitive surface comeinto contact with a location on the touch-sensitive surface thatcorresponds to any of the plurality of user interface elements.Additionally, in some embodiments, the first finger gesture may have amoving point of contact, and the device may be configured to ignorewhether the first finger gesture's moving point of contact intersects orcomes into contact with a location on the touch-sensitive surface thatcorresponds to any user interface element.

Accordingly, in some embodiments, the first finger gesture does notcontact the location on the touch-sensitive surface that corresponds tothe second user interface element (706). Alternatively, detecting thefirst finger gesture is independent of detecting contact by the fingergesture with the location on the touch-sensitive surface thatcorresponds to the second user interface element. Alternatively, thefirst finger gesture remains independent of the location on thetouch-sensitive surface that corresponds to the second user interfaceelement. Alternatively, the first finger gesture may incidentallycontact the location on the touch-sensitive surface that corresponds tothe second user interface element.

In response to detecting the first finger gesture, the device changesthe current focus from the first user interface element in the pluralityof user interface elements to the second user interface element in theplurality of user interface elements, and outputs accessibilityinformation associated with the second user interface element (708).

The outputting or emission of accessibility information associated withuser interface elements can include multiple informational items.Accessibility information may include audible accessibility informationsuch as spoken words or characters, border hit sounds, and any suitablecontextual audible information regarding the user interface oraccessibility aspects of the user interface. Accessibility informationmay include haptic accessibility information such as context-orientedvibrations to convey information about the user interface oraccessibility aspects of the user interface. Accessibility informationmay include sending textual output to Braille displays, large printdisplays, etc. Some or all of these types of accessibility informationmay be used in the same accessibility user interface.

In some embodiments, the accessibility information emitted includes oneor more words outputted as spoken text that corresponds to the seconduser interface element (710) (e.g., the name of the user interfaceelement that the current focus and accessibility cursor were advanced toin response to the first finger gesture). Thus, in the example of FIGS.5A and 5B, the device advances the current focus and accessibilitycursor 502 from texting application icon 141 to photos application icon144 in response to detecting user gesture 504, and the name “photos” maybe outputted as spoken text.

In some embodiments, the spoken text that corresponds to the second userinterface element may include user options (712) (not shown in userinterface FIGS. 5A-5N). For example, if the user interface element maybe manipulated by a user, such as adjusting a volume control, rotating adate adjustment wheel, advancing a play list icon, moving within a textfield, etc., an interaction control phrase may be emitted as spoken textassociated with the user interface element so that a user may know theuser interface element can be adjusted. Non-limiting examples ofinteraction control phrases include “volume control—swipe up toincrease, swipe down to decrease;” “scrubber bar—move right to advance,left to retreat;” “month wheel—swipe up to select previous item, i.e.,decrease towards January, swipe down to select next item, i.e., advancetowards December;” “text field swipe up to move the cursor to the left,swipe down to move the cursor to the right,” etc.

In some embodiments, the accessibility information may include textualaccessibility information (e.g., Braille output, large print display ona screen, etc.) (713)

In some embodiments, the accessibility information may also include aborder hit sound that is played to indicate that the current focus hasmoved from a first user interface element to a second user interfaceelement (714). Thus, in the example of FIGS. 5A and 5B, when the deviceadvances the current focus and accessibility cursor 502 from textapplication user interface element icon 141 to photos application userinterface icon 144 in response to detecting user gesture 504, a borderhit sound may be outputted. As an additional example, if a userinterface has three user interface elements, A, B, and C, where thecurrent focus is on element A, the following sequence may help inform auser about the user's progress in navigating through the user interfaceelements:

-   -   the current focus is on user interface element A;    -   the device detects a gesture, such as the first finger gesture        described above;    -   the device emits a border hit sound;    -   the device changes the current focus from user interface element        A to user interface element B;    -   the device detects a gesture, like the first finger gesture        described above;    -   the device emits another border hit sound; and    -   the device changes the current focus from user interface element        B to user interface element C.

In some embodiments, the device also detects (716) selection of anactivation gesture (i.e., any suitable activation gesture may bedetected, including one or more of the following: pressing home button204 in FIG. 5A, user-initiated gestures on the touch-sensitive surface,shaking the electronic device in a particular pattern, voice-basedactivation, etc.). In response, the device executes (718) the actionassociated with the user interface element that is the current focus.For example, if the current focus is on photos application icon 144(FIG. 5B), which corresponds to the image management application 144,and the device detects selection of the activation gesture, the devicewill execute the image management application. Note that, in someembodiments, if the activation gesture is configured to be performed ona touch-sensitive surface, the activation gesture can be independent ofthe location of the user interface element that has the current focus.Thus, a user need not actually touch or come into contact with aspecific location on the touch-sensitive surface to activate a givenuser interface element.

In some embodiments, the device also detects (720) a second fingergesture on the touch-sensitive surface. The second finger gesture issubstantially similar to the first finger gesture. Like the discussionabove with respect to the first finger gesture, the second fingergesture is independent of contacting a location on the touch-sensitivesurface that corresponds to a third user interface element. In responseto detecting the second finger gesture, the device changes the currentfocus from the second user interface element in the plurality of userinterface elements to the third user interface element in the pluralityof user interface elements, and outputs accessibility informationassociated with the second user interface element (722). For example,see the transition from FIG. 5C to FIG. 5D, where gesture 506 causes thedevice to advance the current focus and the accessibility cursor 502from photos application icon 144 to camera application icon 143.

In some embodiments, the accessibility information associated with thesecond user interface element includes a wrapping sound if the thirduser interface element is either above or below the second userinterface element on the display (724). A wrapping sound is intended tobe a signal to a user that the navigation performed in response to thelast detected user gesture resulted in navigation from a user interfaceelement in a given row (or area of user interface elements) to a userinterface element in a different row (or different area of userinterface elements). For example, see the transition from FIG. 5E toFIG. 5F, where user gesture 510 causes the device to advance the currentfocus and the accessibility cursor 502 from videos application icon 145in the first row of applications 514 to weather application icon 149-1in the next row of application icons 516.

In some embodiments, if the plurality of user interface elements is in ascrollable view, the device scrolls the plurality of user interfaceelements (726). See, e.g., the transition from UI 500I to UI 500J, whichdepicts an example of advancing through a sequence of user interfaceelements displayed on different user interface screens.

In some embodiments, the device detects (728) a third finger gesture onthe touch-sensitive surface. The third finger gesture is independent ofcontacting the location on the touch-sensitive surface that correspondsto the second user interface element. The third finger gesture issubstantially opposite to the first finger gesture.

In response to detecting the third finger gesture, the device changesthe current focus from the third user interface element in the pluralityof user interface elements to the second user interface element in theplurality of user interface elements, and outputs accessibilityinformation associated with the second user interface element (730).

In some embodiments, the device determines if the second user interfaceelement is a first editable text field (e.g., element 530, FIG. 5K), andif so, the device sets a cursor at a first cursor location in the firsteditable text field (e.g., cursor 532, FIG. 5K), and outputsaccessibility information associated with the first editable text field(e.g., output element label 536, “User Name,” FIG. 5K) (732).

In some embodiments, the accessibility information associated with thefirst editable text field may include one or more words output as spokentext which corresponds to the first editable text field (734). Further,in some embodiments, the accessibility information associated with thefirst editable text field may include a text character outputted asspoken text which corresponds to the first character in the firsteditable text field (736).

In some embodiments, the device detects (738) a fourth finger gesture onthe touch-sensitive surface in a direction substantially perpendicularto the first gesture (e.g., downward flick gestures 544 (FIG. 5K) and550 (FIG. 5N) are perpendicular to rightward flick gestures, likegesture 504 in FIG. 5A). In response to detecting the fourth fingergesture, the device determines whether the cursor is at the end of thefirst editable text field (740).

If the cursor is not at the end of the first editable text field (742),the device moves the cursor forward to a second cursor location withinthe first editable text field, and outputs accessibility informationassociated with the second cursor location within the first editabletext field. For example, in response to downward flick gesture 544 (FIG.5K), the device moves the cursor 532 by one character and outputs theletter “C” (FIG. 5L).

In some embodiments, if the current focus is on a text field userinterface element, the device may output audible informationcorresponding to the text field's content on a letter-by-letter basis asa text cursor advances forward or moves backward through the text field.Accordingly, in this embodiment, if a text field contains the word“client” and the user is moving the text cursor from letter-to letterwithin that text field, the device would audibly output the letters “C,”“L,” “I,” “E,” etc. as the user advanced the text cursor from thebeginning towards the end of the text field (see, e.g., FIGS. 5K and 5Lwhere the text cursor 532 advances from the position preceding the firsttext string 534 “client” to the position between the “c” and the “l” inthe first text string 534).

If the cursor is at the end of the first editable text field (744),however, the device maintains the cursor at the end of the firsteditable text field, and outputs accessibility information indicatingthat the cursor is at the end of the first editable text field (744).For example, in response to downward flick gesture 550 (FIG. 5N) thedevice keeps the cursor at the end of the editable text field in element530 and outputs accessibility information indicating that the cursor isat the end of editable text field in element 530.

In some embodiments, the device detects (746) a fifth finger gesture onthe touch-sensitive surface in a direction substantially opposite to thefourth gesture when the cursor is in the first editable text field. Inresponse to detecting the fifth finger gesture, the device moves thecursor backward in the first editable text field to a third cursorlocation within the first editable text field, and outputs accessibilityinformation associated with the third cursor location within the firsteditable text field (748). For example, see the transition from FIG. 5Lto FIG. 5M, where in the text cursor 532 moves from the position betweenthe “c” and the “l” in the first text string 534 to the positionpreceding the first text string 534 “client” after detecting usergesture 546. In this example, the third cursor position is the same asthe first cursor position.

Note that method 700 navigates among user interface elements using afirst type of finger gesture (e.g., a swipe gesture 504) whose detectionand interpretation is independent of contacting a location on thetouch-sensitive surface that corresponds to a particular user interfaceelement. Method 700 may complement or replace existing accessibilitymethods. For example, other types of finger gestures (e.g., a tapgesture, a tap-and-hold gesture, or finger movement below a predefinedspeed on the touch sensitive surface) may also be used in anaccessibility interface, where the detection and interpretation of theseother types of gestures does depend on contacting a location on thetouch-sensitive surface that corresponds to a particular user interfaceelement. For example, see the discussion of gesture 519 in FIG. 5Habove.

FIGS. 8A-8B are flow diagrams illustrating an accessibility method forcommunicating document section indicia in accordance with someembodiments. The method 800 is performed at an electronic device such asa multifunction device (e.g., 300, FIG. 3) with a display and atouch-sensitive surface in conjunction with accessibility module 129.Some operations in method 800 may be combined and/or the order of someoperations may be changed.

In some embodiments, the touch-sensitive surface is separate from thedisplay. In some embodiments, the touch-sensitive surface is a touch pad(e.g., 355, FIG. 3).

In some embodiments, the display is a touch screen display (e.g., 112,FIG. 2) and the touch-sensitive surface is on the display (822). In someembodiments, the method is performed at a portable multifunction devicewith a touch screen display (e.g., portable multifunction device 100).

As described below, the method 800 provides an efficient way to navigatethough large electronic documents and provide accessibility informationabout the current location in a large document. The method reduces thecognitive burden on a user with impaired vision when using an electronicdevice with a display and a touch-sensitive surface to navigate thoughlarge documents, thereby creating a more efficient human-machineinterface. For battery-operated devices, enabling a user to navigatefaster and more efficiently conserves power and increases the timebetween battery charges.

The device displays (802) a first section of a document on the display,wherein the document has a plurality of sections (see, e.g., FIG. 5P, UI500P depicting the beginning of the document 553-1 being displayedwithin document display area 552). The document types that can bedisplayed with the methods disclosed herein include, without limitation,memos, web pages, HTML documents, XML documents, tables, spreadsheets,charts, emails, forms, correspondence, e-books, pdfs, etc.

The device also outputs (803) an audible document section indicia thatcorresponds to the first section of the document. For instance, in theFIG. 5P example, the audible document section indicia may be a spokenmessage such as “section 1 of 4 of document being displayed.” When atable, spreadsheet, chart, or similar tabular-formatted document isbeing displayed, audible document section indicia may indicate relativelocation within the tabular-formatted document, e.g., “displaying rows1-5 of 25 rows,” or “displaying columns A-E of 25 columns,” or“displaying columns A-E of columns A-Y.”

The device detects (804) a first finger gesture on the touch-sensitivesurface (e.g. user gesture 555 in FIG. 5P). In some embodiments, thefirst finger gesture includes multiple concurrent finger contacts on thetouch-sensitive surface (806) (see e.g., multi-finger gesture 555 inFIG. 5P).

In response to detecting the first finger gesture, the device ceases todisplay the first section of the document and instead displays a secondsection of the document on the display, wherein the second section ofthe document is adjacent to the first section of the document, andoutputs an audible document section indicia that corresponds to thesecond section of the document (808). For example, see the transitionfrom FIG. 5P to FIG. 5Q, where in FIG. 5Q, the user interface has ceasedto display the beginning of the document 553-1, and has insteaddisplayed a second section of the document 553-2 within document displayarea 552. Section 553-2 is adjacent to section 553-1 in the document553.

In some embodiments, the audible document section indicia includes oneor more words outputted as spoken text that provide navigationinformation about a currently displayed section of the document (810).For instance, in the FIG. 5Q example, the audible document sectionindicia may be a spoken message such as “section 2 of 4 of documentbeing displayed.” When a table, spreadsheet, chart, or similartabular-formatted document is being displayed, audible document sectionindicia may indicate relative location within the tabular-formatteddocument, e.g., “displaying rows 6-10 of 25 rows,” or “displayingcolumns F-J of 25 columns,” or “displaying columns F-J of columns A-Y.”

In some embodiments, the device detects (812) a second finger gesture onthe touch-sensitive surface while displaying the second section of thedocument. In response to detecting the second finger gesture on thetouch-sensitive surface, the device ceases to display the second sectionof the document, displays a third section of the document on thedisplay, wherein the third section of the document is adjacent to thesecond section of the document, and outputs a third audible documentsection indicia that corresponds to the third section of the document(814). For example, see FIG. 5Q where the device detects user gesture556. In response, in FIG. 5R, the device displays a third section of thedocument 553-3 within document display area 552. Section 553-3 isadjacent to section 553-2 in the document 553. See also, FIG. 5R wherethe device detects multi-finger gesture 557, and in response, in FIG.5S, the device displays a fourth section of the document 553-4 withinthe document display area 552.

In some embodiments, the audible document section indicia thatcorresponds to the third section of the document includes one or morewords outputted as spoken text that corresponds to the third section ofthe document (816). For instance, in the FIG. 5R example, the audibledocument section indicia that corresponds to the third section of thedocument may be a spoken message such as “section 3 of 4 of documentbeing displayed.” In the FIG. 5S example, the audible document sectionindicia that corresponds to the fourth section of the document may be aspoken message such as “displaying document section 4 of 4.”

In some embodiments, if the end of the document is being displayed, anaudible document section indicia may include one or more words to thateffect as a spoken message, such as “end of displayed document,” or“displaying final document section.” See, FIG. 5S, where the fourthsection of the document 553-4 is displayed within document display area552 and is the end of the document.

In some embodiments, the device detects (818) a third finger gesture onthe touch-sensitive surface while displaying the third section of thedocument. The third finger gesture is in a direction substantiallyopposite the second finger gesture (e.g., user gesture 558 in FIG. 5S issubstantially opposite gesture 557 in FIG. 5R). In response to detectingthe third finger gesture on the touch-sensitive surface, the deviceceases to display the third section of the document, redisplays thesecond section of the document on the display, and outputs the audibledocument section indicia that corresponds to the second section of thedocument (820). For example, FIG. 5T depicts the user interface againdisplaying section 553-3 of the document in response to detectinggesture 558. In some embodiments, the second audible document sectionindicia, if output more than once, may include an additional message,e.g. “redisplaying section 3 of 4 of document being displayed.” In someembodiments, if a user gesture returns the user to the top of thedocument being displayed, an audible document section indicia may beoutput that indicates the user is at the top of the document, e.g.“redisplaying beginning of document.”

FIGS. 9A-9B are flow diagrams illustrating an accessibility method thatuses location-independent gestures to adjust user interface navigationsettings in accordance with some embodiments. The method 900 isperformed at an electronic device such as a multifunction device (e.g.,300, FIG. 3) with a display and a touch-sensitive surface in conjunctionwith accessibility module 129. Some operations in method 900 may becombined and/or the order of some operations may be changed.

In some embodiments, the touch-sensitive surface is separate from thedisplay. In some embodiments, the touch-sensitive surface is a touch pad(e.g., 355, FIG. 3).

In some embodiments, the display is a touch screen display (e.g., 112,FIG. 2) and the touch-sensitive surface is on the display (926). In someembodiments, the method is performed at a portable multifunction devicewith a touch screen display (e.g., portable multifunction device 100).

As described below, the method 900 provides an efficient way to uselocation-independent gestures to adjust user interface navigationsettings and provide accessibility information about the settings. Themethod reduces the cognitive burden on a user with impaired vision whenadjusting navigation settings on an electronic device with a display anda touch-sensitive surface, thereby creating a more efficienthuman-machine interface. For battery-operated devices, enabling a userto adjust navigation settings faster and more efficiently conservespower and increases the time between battery charges.

The device displays (902) a plurality of user interface elements on thedisplay.

The device detects (904) a first user interface navigation gesture by afinger on the touch-sensitive surface (e.g., user gesture 544 in FIG.5K). In response to detecting the first user interface navigationgesture by the finger on the touch-sensitive surface, the devicenavigates (906) in the plurality of user interface elements inaccordance with a current navigable unit type, wherein the currentnavigable unit type is set to a first navigable unit type selected froma plurality of navigable unit types. For example, between FIG. 5K andFIG. 5L, the text cursor 532 has advanced one character, i.e., from theposition preceding first text string 534 within user name text entryelement 530 to the position immediately after the “c” of the string“client.” In FIGS. 5K and 5L, the current navigable unit type is bycharacter, rather than by word.

In some embodiments, the plurality of navigable unit types may includecharacters, words, sentences, lines, paragraphs, pages, visited links,unvisited links, and/or any other context appropriate navigable unit asdescribed above (908). Further, accessibility information about anavigable unit type may be spoken words such as “characters,” “words,”“sentences,” “lines,” “paragraphs,” “pages,” “visited links,” “unvisitedlinks,” or any words describing other context appropriate navigableunits as described above (910).

The device detects (912) a first user interface navigation settinggesture on the touch-sensitive surface, wherein the first user interfacenavigation setting gesture is independent of contacting a location onthe touch-sensitive surface that corresponds to any of the plurality ofuser interface elements (e.g., rotor gesture 562 in FIG. 5U). In otherwords, it is immaterial whether the gesture contacts or does not contacta location on the touch-sensitive surface that corresponds to any userinterface element.

In some embodiments, the first user interface navigation setting gestureis a multi-finger gesture (914) (e.g., rotor gesture 562 in FIG. 5U,which includes a first and second finger contact, 562-1 and 562-2,respectively). In some embodiments, the first user interface navigationsetting gesture is a two-finger rotation gesture (915) (e.g., rotorgesture 562 in FIG. 5U).

In some embodiments, the first user interface navigation setting gestureincludes a first finger contact and a concurrent second finger contactthat are separated by a distance, and the distance between the firstfinger contact and the second finger contact includes a center point.The first finger contact and the second finger contact rotate on thetouch-sensitive surface around the center point (916). For example, seerotor gesture 562 in FIG. 5U, which includes first and second fingercontacts (562-1 and 562-2) that rotate (562-3 and 562-4) on thetouch-sensitive surface around a virtual center point between the firstand second finger contacts. As noted above, this gesture simulates theturning of an invisible dial, rotor, or control.

Rotation associated with user interface navigation setting gestures maybe in a clockwise/counter-clockwise fashion. But depending on designconsiderations and the user interface options to change, in someembodiments, user interface navigation setting gestures may also beperformed in a right/left or up/down orientation whenever needed orappropriate.

In response to detecting the first user interface navigation settinggesture on the touch-sensitive surface, the device changes the currentnavigable unit type from the first navigable unit type to a secondnavigable unit type selected from the plurality of navigable unit types,and outputs accessibility information about the second navigable unittype (918).

After changing the current navigable unit type from the first navigableunit type to the second navigable unit type, the device detects (920) asecond user interface navigation gesture by the finger on thetouch-sensitive surface. The second user interface navigation gesture issubstantially the same as the first user interface navigation gesture(see, e.g., user gesture 564 in FIG. 5V, which is the same type ofgesture as gesture 544 in FIG. 5K, namely a downward swipe or flickgesture). In response to detecting this second user interface navigationgesture by the finger on the touch-sensitive surface, the devicenavigates (922) in the plurality of user interface elements inaccordance with the current navigable unit type, wherein the currentnavigable unit type is set to the second navigable unit type. Forexample, the location of the text cursor 532 in FIG. 5W has movedforward one word, i.e., from the beginning of the text string “client”to the beginning of the name “li,” in accordance with the currentnavigable unit type (i.e., word-by-word navigation).

In some embodiments, the device may then detect (924) a second userinterface navigation setting gesture on the touch-sensitive surface. Thesecond user interface navigation setting gesture is independent ofcontacting a location on the touch-sensitive surface that corresponds toany of the plurality of user interface elements. The first userinterface navigation setting gesture has an associated direction (e.g.,rotor gesture 562 in FIG. 5U is in a counterclockwise direction) and thesecond user interface navigation setting gesture has an associateddirection that is substantially opposite the direction of the first userinterface navigation setting gesture (e.g., rotor gesture 566 in FIG. 5Xis in a clockwise direction). In response to detecting the second userinterface navigation setting gesture, the device changes the currentnavigable unit type from the second navigable unit type (e.g., words) tothe first navigable unit type (e.g., characters) and outputsaccessibility information about the first navigable unit type (e.g.,outputting “characters”). See, also FIGS. 5Y and 5Z, where as a resultof rotor gesture 566, the current navigable unit type is set tocharacter and gesture 568 moves the text cursor 532 forward onecharacter to the position between “l” and “i” in the word “li.”.

In some embodiments, the current navigable unit type iscontext-dependent, and a current focus is on a user interface element inthe plurality of user interface elements (928). In those embodiments,the device changes (930) the current navigable unit type based on a typeof the user interface element in the plurality of user interfaceelements that has the current focus. For example, while a user can movelink-by-link in a web page depending on the rotor setting, thisapplication behavior would make no sense in a stopwatch time keepingapplication. Thus, the accessibility user interface changes the currentnavigable unit type to an appropriate setting for the user interfaceelement with the current focus, taking into account the user applicationthat the user is interacting with at that time.

FIGS. 10A-10C are flow diagrams illustrating an accessibility method fornavigating via user interface containers in accordance with someembodiments. The method 1000 is performed at an electronic device suchas a multifunction device (e.g., 300, FIG. 3) with a display and atouch-sensitive surface in conjunction with accessibility module 129.Some operations in method 1000 may be combined and/or the order of someoperations may be changed.

In some embodiments, the touch-sensitive surface is separate from thedisplay. In some embodiments, the touch-sensitive surface is a touch pad(e.g., 355, FIG. 3).

In some embodiments, the display is a touch screen display (e.g., 112,FIG. 2) and the touch-sensitive surface is on the display. In someembodiments, the method is performed at a portable multifunction devicewith a touch screen display (e.g., portable multifunction device 100).

As described below, the method 1000 provides an efficient way to usecontainers to navigate though complex visual displays. The methodreduces the cognitive burden on a user with impaired vision when usingan electronic device with a display and a touch-sensitive surface,thereby creating a more efficient human-machine interface. Forbattery-operated devices, enabling a user to navigate faster and moreefficiently conserves power and increases the time between batterycharges.

The device maps (1001) at least a first portion of the display to thetouch-sensitive surface (e.g., window 612, FIG. 6B; window 634, FIG.6H).

The device concurrently displays (1002) a plurality of user interfacecontainers on the display (e.g., window 612 includes a plurality ofcontainers, FIG. 6B; window 634 includes a plurality of containers, FIG.6H).

In some embodiments, the plurality of user interface containerscomprises a hierarchy of user interface containers (1004) For example,see UI 600B-UI 600D, where the three areas discussed form a hierarchy.Window 612 is at a first level, the area of window 612 depicted withcurrent container indicator 614-2 in UI 600C is at a second, lowerlevel, and the area of window 612 within current container indicator614-3 in UI 600D is at a third, still lower level in the hierarchy.

The device detects (1006) a user interface container selection eventthat selects a first user interface container in the plurality of userinterface containers (e.g., user interface container selection event 616in UI 600B selects the area of window 612 depicted with currentcontainer indicator 614-2 in UI 600C). Any suitable gesture may be usedfor user interface container selection events. In some embodiments, aspecific gesture on the touch-sensitive surface, such as a two-fingerswipe gesture 616, is used for this purpose.

In some embodiments, the device outputs (1008) accessibility informationassociated with the first user interface container in response todetecting the user interface container selection event. For example, inFIG. 6C, the audible information associated with the container outlinedby indicator 614-2 may be the spoken words “interacting with container,three items.”

In some embodiments, the device visually highlights (1010) the firstuser interface container in response to detecting the user interfacecontainer selection event (e.g., current container indicator 614-2 inFIG. 6C). In some embodiments, a dark border indicates that a given userinterface container is the currently selected container.

In some embodiments, the first user interface container includes aplurality of user interface elements (1012) (e.g., in FIG. 6C, thecontainer outlined by indicator 614-2 includes two buttons and ascrolling text box).

In some embodiments, multiple UI elements in the plurality of userinterface elements are substantially aligned in one direction within thefirst user interface container (1014) For example, in FIG. 6A-1, userinterface elements 604-1, 604-2, and 604-3 are horizontally alignedwithin container 604. As another example, in UIs 600P-600S (FIGS.6P-6S), lines 670-1 through 670-6 each have an axis 671 along which theuser interface elements (words) are aligned.

In response to detecting the user interface container selection event,the device ceases to map the first portion of the display to thetouch-sensitive surface, and proportionally maps (1016) the first userinterface container to be coextensive or substantially coextensive withthe touch-sensitive surface. For example, in response to gesture 616(FIG. 6B), mapping 620 (FIG. 6C) replaces mapping 615 (FIG. 6B).

In some embodiments, the mapping of a user interface container totouch-sensitive surface 603 is a proportional mapping where the userinterface container is coextensive or substantially coextensive with thetouch-sensitive surface. As used herein, coextensive means that theentire area of the user interface container is proportionally mapped tothe entire the area of the touch-sensitive surface, i.e., 100% of theuser interface container is proportionally mapped to 100% of thetouch-sensitive surface that is operable to accept touch-based inputfrom a user. As used herein, substantially coextensive means that theproportional mapping maps the user interface container to a substantialportion (e.g., 80% to 100%) of the touch-sensitive surface.

In some embodiments, the mapping of a user interface container to besubstantially coextensive with the touch-sensitive surface means that acontainer has a height and a width, and substantially all of the area ofthe container is mapped to the touch-sensitive surface by scaling theheight of the container to the height of the touch-sensitive surface,and scaling the width of the container to the width of thetouch-sensitive surface.

The mapping of a user interface container to be substantiallycoextensive with the touch-sensitive surface confines accessiblenavigation on the display to the interior of a currently selected userinterface container, with an area of the touch-sensitive surface scaledto the area of the currently selected user interface container. Becausethe mapped touch-sensitive surface represents only a currently selectedcontainer and not the entire display, a user cannot stray into otherareas of the display by accident.

In some embodiments, while the first user interface container isproportionally mapped to be substantially coextensive with thetouch-sensitive surface, the device detects a first user interfaceelement navigation gesture on the touch-sensitive surface, wherein thefirst user interface element navigation gesture contacts an area of thetouch-sensitive surface to which at least one user interface element ofthe plurality of user interface elements is mapped. In response todetecting the first user interface element navigation gesture, thedevice also outputs accessibility information associated with the atleast one user interface element of the plurality of user interfaceelements (1018). For example, as a user navigates over locations on thetouch-sensitive surface that correspond to user interface elements inthe container, the accessible user interface will provide audibleinformation about the corresponding user interface elements. Forexample, see the discussions above of user interface navigation gesture629 in FIGS. 6E and 6F and user interface navigation gesture 640 inFIGS. 6I and 6J.

In some embodiments, while the first user interface container isproportionally mapped to be substantially coextensive with thetouch-sensitive surface, the device detects a second user interfaceelement navigation gesture on the touch-sensitive surface, whereindetecting the second user interface element navigation gesture includesdetecting a finger contact, and detecting movement of the finger contactin substantially one direction. In response to detecting the movement ofthe finger contact in substantially one direction, the device moves acurrent focus in the first user interface container in accordance withthe movement of the finger contact, wherein locations for the currentfocus are substantially confined within a first axis aligned with themultiple user interface elements (1020). For example, see the discussionabove of user interface navigation gesture 674 in FIGS. 6Q and 6R.

Further, in these embodiments, if the device detects movement of thefinger contact that substantially departs from the one direction (e.g.,moves in a direction that is greater than a predetermined angle (e.g.,30, 40, or 45° from the one direction), in response, the device willmove the current focus away from a user interface element within thefirst aligned axis to a user interface element in the first userinterface container other than one of the multiple user interfaceelements that are along the first aligned axis (1022). For example, seethe discussion above of user interface navigation gesture 676 in FIGS.6R and 6S. In some embodiments, the device detects movement of thefinger contact that substantially departs from the one direction whenthe device detects movement greater than a predefined amount (e.g., 8mm) in a direction perpendicular to the one direction.

In some embodiments, while the first user interface container isproportionally mapped to be substantially coextensive with thetouch-sensitive surface, the device detects another user interfacecontainer selection event that selects a second user interface containerin the plurality of user interface containers. In response to detectingthe another user interface container selection event, the device ceasesto proportionally map the first user interface container to besubstantially coextensive with the touch-sensitive surface, andproportionally maps the second user interface container to besubstantially coextensive with the touch-sensitive surface (1024). Thesecond user interface container is in a level of the hierarchy of userinterface containers that is immediately below the first user interfacecontainer. For example, in response to gesture 618 (FIG. 6C), mapping624 (FIG. 6D) replaces mapping 620 (FIG. 6C).

In some embodiments, if the device detects a user interface containernavigation event that deselects the first user interface container, thedevice ceases to proportionally map the first user interface containerto be substantially coextensive with the touch-sensitive surface, andremaps the first portion of the display to the touch-sensitive surface(1026). For example, in response to detecting user interface containerdeselection event 628 (FIG. 6F), the user interface containerhighlighted by current container indicator 614-4 is remapped totouch-sensitive surface 603 in FIG. 6F, as it had been mapped 625 inFIG. 6E. Similarly, if gesture 618 in FIG. 6C was a two-finger swipegesture to the left (rather than a two-finger swipe gesture to theright), then mapping 615 (FIG. 6B) would have replaced mapping 620 (FIG.6C).

In some embodiments, the device traverses down the hierarchy of userinterface containers in response to detecting a user interface containerselection event (1028). In some embodiments, the user interfacecontainer selection event is a multi-finger gesture (1030). In someembodiments, the user interface container selection event is amulti-finger depinching gesture (1032). In some embodiments, the userinterface container selection event is a multi-finger swipe gesture(1034). For example, as discussed above, the device traverses down thehierarchy of containers in response to two-finger swipe gestures to theright 616 (FIG. 6B), 618 (FIG. 6C), and 622 (FIG. 6D).

In some embodiments, the device traverses up the hierarchy of userinterface containers in response to detecting a user interface containerdeselection event (1036). In some embodiments, the user interfacecontainer deselection event is a multi-finger gesture (1038). In someembodiments, the user interface container selection event is amulti-finger pinching gesture (1040). In some embodiments, the userinterface container selection event is a multi-finger swipe gesture(1042). For example, as discussed above, the device traverses up thehierarchy of containers in response to two-finger swipe gesture to theleft 628 (FIG. 6F).

FIGS. 11A-11B are flow diagrams illustrating an accessibility method forperforming magnification about a point in a multi-finger bounding box inaccordance with some embodiments. The method 1100 is performed at anelectronic device such as a multifunction device (e.g., 300, FIG. 3)with a display and a touch-sensitive surface in conjunction withaccessibility module 129. Some operations in method 1100 may be combinedand/or the order of some operations may be changed.

In some embodiments, the display is a touch screen display (e.g., 112,FIG. 2) and the touch-sensitive surface is on the display. In someembodiments, the method is performed at a portable multifunction devicewith a touch screen display (e.g., portable multifunction device 100).

As described below, the method 1100 provides an efficient way to magnifyabout any point on the touch screen display in response to amulti-finger gesture, including points at the edges or corners of thedisplay. The method reduces the cognitive burden on a user with impairedvision when using an electronic device with a touch screen display,thereby creating a more efficient human-machine interface. Forbattery-operated devices, enabling a user to navigate faster and moreefficiently conserves power and increases the time between batterycharges.

The device displays (1102) content in a user interface at a firstmagnification on the touch screen display. In some embodiments, thefirst magnification corresponds to a default unmagnified view (1104)(see, e.g., FIG. 4A).

The device detects (1106) a first multi-finger gesture on the touchscreen display, wherein detecting the first multi-finger gestureincludes detecting concurrent multi-finger contacts on the touch screendisplay. (See, e.g., FIG. 5OO, where multi-finger gesture 591 isdetected with three concurrent finger contacts 591-1 a, 591-1 b, and591-1 c on the touch screen 112.)

In some embodiments, the first multi-finger gesture is a three-fingergesture (1108). In some embodiments, the first multi-finger gesture is athree-finger, double-tap gesture (1110). (See, e.g., FIG. 5OO, wheremulti-finger gesture 591 is detected with three concurrent fingercontacts 591-1 a, 591-1 b, and 591-1 c on the touch screen 112, and mayrepresent a three-finger double-tap gesture.)

The device determines (1112) a bounding box for the concurrentmulti-finger contacts. In some embodiments, the bounding box is arectangular bounding box (1114). (See, e.g., FIG. 5OO, where rectangularbounding box 592-1 is determined for concurrent multi-finger contacts591-1 a, 591-1 b, and 591-1 c.)

In response to detecting the first multi-finger gesture on the touchscreen display, the device magnifies (1116) the content in the userinterface about a point in the bounding box to a second magnification,greater than the first magnification, on the touch screen display. Forexample, multi-finger gesture 591 (FIG. 5OO) may magnify the content inUI 500II (FIG. 5II) to a second magnification (e.g., UI 500B, FIG. 5JJ).The point in the bounding box is at a default position (e.g., acentroid) within the bounding box when the bounding box is located atthe center of the touch screen display (see, e.g., FIG. 5OO point 593-1a at the centroid of bounding box 592-1). When the bounding box islocated away from the center of the touch screen display, the point inthe bounding box is displaced from the default position within thebounding box towards a respective edge of the bounding box by an amountthat corresponds to a displacement of the bounding box from the centerof the touch screen display towards a corresponding edge of the touchscreen display. (see, e.g., FIG. 5OO bounding box 592-2, which is closerto the right edge of touch screen 112, so point 593-1 b is closer to theright edge of the bounding box 592-2)

In some embodiments, magnifying the content in the user interface aboutthe point in the bounding box includes displaying (1118) a zooming-inanimation about the point in the bounding box.

In some embodiments, the point in the bounding box is displaced from acentroid of the bounding box towards a respective edge of the boundingbox by an amount that increases as a distance that the bounding box isdisplaced from the center of the touch screen display towards acorresponding edge of the touch screen display increases (1120)(compare, e.g., FIG. 5OO bounding box 592-2, which is closer to theright edge of touch screen 112 than bounding box 592-1, so point 593-1 bis closer to the right edge of the bounding box 592-2, with FIG. 5OObounding box 592-3, which is even closer to the right edge of touchscreen 112 than bounding box 592-2, so point 593-1 c is even closer tothe right edge of the bounding box 592-3 than point 593-1 b was to theedge of bounding box 592-2).

In some embodiments, the point in the bounding box is displaced from thepoint in the bounding box towards the respective edge of the boundingbox by an amount that is proportional to the distance that the boundingbox is displaced from the center of the touch screen display towards thecorresponding edge of the touch screen display (1122) (compare, e.g.,within FIG. 5OO, the point 593-1 b in bounding box 592-2, where thebounding box 592-2 is closer to the center of touch screen 112, withFIG. 5OO point 593-1 c within bounding box 592-3, which is farther fromthe center of touch screen 112 than bounding box 592-2, so point 593-1 cis proportionally closer to the right edge of the bounding box 592-3than point 593-1 b was to the edge of bounding box 592-2).

In some embodiments, the point in the bounding box is displaced from thepoint in the bounding box to the respective edge of the bounding boxwhen the respective edge of the bounding box is located at acorresponding edge of the touch screen display (1124) (see, e.g., FIG.5OO point 593-1 c within bounding box 592-3; FIG. 5OO point 593-1 ewithin bounding box 592-5; and FIG. 5PP point 593-1 h within boundingbox 592-8).

In some embodiments, while displaying the content in the user interfaceat the second magnification on the touch screen display, the devicedetects a second multi-finger gesture on the touch screen display. Inresponse to detecting the second multi-finger gesture on the touchscreen display, the device demagnifies the content in the user interfaceto the first magnification (1126). For example, another multi-fingergesture 591 (FIG. 5OO) may demagnify the content in UI 500JJ (FIG. 5JJ)to the first magnification (e.g., UI 500II, FIG. 5II). In theseembodiments, any of the following may be implemented as well: the secondmulti-finger gesture is a three-finger gesture (1128); the secondmulti-finger gesture is a three-finger, double-tap gesture (1130); thesecond multi-finger gesture is the same or substantially the same as thefirst multi-finger gesture (1132); and/or demagnifying the content inthe user interface includes displaying a zooming-out animation (1134).

FIGS. 12A-12B are flow diagrams illustrating an accessibility method forsingle-finger push panning in accordance with some embodiments. Themethod 1200 is performed at an electronic device such as a multifunctiondevice (e.g., 300, FIG. 3) with a display and a touch-sensitive surfacein conjunction with accessibility module 129. Some operations in method1200 may be combined and/or the order of some operations may be changed.

In some embodiments, the display is a touch screen display (e.g., 112,FIG. 2) and the touch-sensitive surface is on the display. In someembodiments, the method is performed at a portable multifunction devicewith a touch screen display (e.g., portable multifunction device 100).

As described below, the method 1200 provides an efficient way to useboth multi-finger and single finger gestures to scroll content. Themethod reduces the cognitive burden on a user with impaired vision whenusing an electronic device with a touch screen display, thereby creatinga more efficient human-machine interface. For battery-operated devices,enabling a user to navigate faster and more efficiently conserves powerand increases the time between battery charges.

The device displays (1202) content in a user interface on the touchscreen display. In some embodiments, displaying the content in the userinterface on the touch screen display comprises displaying a magnifiedview of the content in the user interface (1204). (See, e.g., FIG. 5JJ.)

The device detects (1206) a plurality of concurrent finger contacts onthe touch screen display. In some embodiments, the plurality ofconcurrent finger contacts are three concurrent finger contacts (1208)(See, e.g., three-finger movement gesture 582, FIG. 5JJ.)

The device detects (1210) movement of the plurality of concurrent fingercontacts on the touch screen display. (See, e.g., three-finger movementgesture 582, FIG. 5JJ.)

The device scrolls (1212) the content in the user interface inaccordance with the movement of the plurality of concurrent fingercontacts (e.g., the device performs a two-dimensional translation of theuser interface that tracks the movement of the centroid of the pluralityof concurrent finger contacts, or alternatively tracks the movement ofthe centroid of one finger contact in the plurality of concurrent fingercontacts). (See, e.g., in response to three-finger movement gesture 582depicted in FIG. 5JJ, the accessibility user interface panned so thatinstead of displaying user interface application icons Stocks 149-2,Voice Memo 142, Alarm 149-4, and Dictionary 149-5, the accessibilityuser interface displays within FIG. 5KK application icons Photos 144 andCamera 143 in the upper portion of the display, and Stocks 149-2 andVoice Memo 142 in the lower portion of the display.)

In some embodiments, before detecting lift off of all but one of theplurality of concurrent finger contacts, the device detects lift off ofsome of the plurality of concurrent finger contacts while continuing todetect at least two remaining finger contacts (e.g., for threeconcurrent finger contacts, the device detects lift off of one of thethree concurrent finger contacts). The device detects movement of the atleast two remaining finger contacts on the touch screen display; and thedevice scrolls the content in the user interface in accordance with themovement of the at least two remaining finger contacts (e.g.,two-dimensional translation of the user interface that tracks themovement of the centroid of the at least two remaining finger contactsor that tracks the movement of the centroid of one finger contact in theat least two remaining finger contacts) (1214).

The device detects (1216) lift off of all but one of the plurality ofconcurrent finger contacts while continuing to detect a single remainingfinger contact (see, e.g., in FIG. 5QQ the device detects lift off ofall but one of the plurality of concurrent finger contacts 597-1). Forexample, for three concurrent finger contacts, the single remainingfinger contact is the one of the three concurrent finger contacts thatremains after lift off of the other two contacts is detected.

The device ceases (1218) scrolling of the content in the user interfacewhen the single remaining finger contact is located in a predefinedcentral portion of the touch screen display (e.g., the middle 60% of thetouch screen display) (See, e.g., in FIG. 5QQ, the user interface ceasedscrolling of the content in the user interface when the single remainingfinger contact 597-1 is located in a predefined central portion 598 ofthe touch screen 112.)

The device scrolls (1220) the content in the user interface when thesingle remaining finger contact is located in a predefined borderportion of the touch screen display (e.g., the remaining 40% of thetouch screen display between the predefined central portion and theedges of the touch screen display). (See, e.g., in FIG. 5QQ, the singleremaining finger contact 597-2 is located near the interior edge 599-1of predefined border portion 599, which results in scrolling of thecontent in the user interface away from the edge of the touch screen112.)

In some embodiments, the device scrolls (1222) the content in the userinterface when the single remaining finger contact is located in thepredefined border portion of the touch screen display, includingscrolling the content in the user interface away from an edge of thetouch screen display that is nearest to the single remaining fingercontact (e.g., translating the user interface from left-to-right if thesingle remaining finger contact is nearest to the left edge of the touchscreen display; translating the user interface from right-to-left if thesingle remaining finger contact is nearest to the right edge of thetouch screen display; translating the user interface from top-to-bottomif the single remaining finger contact is nearest to the top edge of thetouch screen display; or translating the user interface frombottom-to-top if the single remaining finger contact is nearest to thebottom edge of the touch screen display). To wit, if the singleremaining finger contact is nearest to the left edge of the screen, thecontent in the user interface will scroll toward the right edge of thescreen, revealing user interface content that was beyond the left edgeof the touch screen display's edge.

In some embodiments, the scrolling of the content in the user interfaceaway from the edge of the touch screen display that is nearest to thesingle remaining finger contact has a scrolling speed that inverselycorresponds to a distance between the single remaining finger contactand the edge of the touch screen display that is nearest to the singleremaining finger contact (1224) (i.e., the scrolling speed increases asthe single remaining finger contact moves closer to the edge of thetouch screen display that is nearest to the single remaining fingercontact). (See, e.g., in FIG. 5QQ, where single remaining finger contact597-2 will scroll the content left-to-right in the user interface slowerthan single remaining finger contact 597-3 will scroll the contentleft-to-right in the user interface.)

In some embodiments, the increase in scrolling speed is based on theposition of the single remaining finger contact in the predefined borderportion of the touch screen display, rather than the speed of movementof the single remaining finger contact.

In some embodiments, after detecting lift off of all but one of theplurality of concurrent finger contacts: the device detects lift off ofthe single remaining finger contact in the predefined border portion ofthe touch screen display, and, in response to detecting lift off of thesingle remaining finger contact in the predefined border portion of thetouch screen display, the device ceases to scroll the content in theuser interface (1226).

Common Core Gesture Set

In some embodiments, some or all of a set of gestures described abovemay be used on accessible electronic devices with differing formfactors. For example, a group of gestures for voice over navigation,such as location-independent flick/swipe gestures, navigation settingsgestures (e.g., rotor gestures), and split gestures may be implementedas part of accessible user interfaces for desktop computers with trackpads and/or touch screens, laptop computers with track pads and/or touchscreens, tablet computers with track pads and/or touch screens, andportable multifunction devices with track pads and/or touch screens(e.g., handheld devices such as the iPhone® and iPod Touch® devices fromApple, Inc. of Cupertino, Calif.). The group of gestures that are usedin all of these form factors forms a common core gesture set foraccessible devices.

Such a common core gesture set provides a consistent, easy-to-learninterface that works in the same manner across multiple device formfactors. This commonality reduces the cognitive burden on users,especially users with physical or learning disabilities, because theusers do not have to learn a new user interface from scratch for eachtype of accessible device that they use.

In some embodiments, a common core gesture set for accessibility isembodied in a computer readable storage medium having executableinstructions stored therein. The executable instructions includeinstructions for recognizing a common core gesture set foraccessibility. When the instructions are executed by an electronicdevice with a touch-sensitive surface, the instructions cause the deviceto recognize and respond to gestures selected from the common coregesture set. The computer readable storage medium is configured to beinstalled in a plurality of accessible device form factors, includingtwo or more of: desktop computers with track pads and/or touch screens,laptop computers with track pads and/or touch screens, tablet computerswith track pads and/or touch screens, and portable multifunction deviceswith track pads and/or touch screens.

In some embodiments, the instructions for recognizing the common coregesture set for accessibility are configured to be stored in a pluralityof computer readable storage medium types, such as high-speed randomaccess memory, flash memory, magnetic storage media, optical storagemedia, and other types of non-volatile memory.

The steps in the information processing methods described above may beimplemented by running one or more functional modules in informationprocessing apparatus such as general purpose processors or applicationspecific chips. These modules, combinations of these modules, and/ortheir combination with general hardware (e.g., as described above withrespect to FIGS. 1A, 1B and 3) are all included within the scope ofprotection of the invention.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method, comprising: at an accessible electronicdevice with a touch-sensitive surface and a display: displaying aplurality of user interface elements on the display, wherein a currentfocus is on a first user interface element; detecting a first fingergesture directly contacting on the touch-sensitive surface, wherein thefirst finger gesture is remote from a location on the touch-sensitivesurface that corresponds to a second user interface element; and, inresponse to detecting the first finger gesture: changing the currentfocus from the first user interface element in the plurality of userinterface elements to the second user interface element in the pluralityof user interface elements; and outputting audible accessibilityinformation associated with the second user interface element.
 2. Themethod of claim 1, further comprising: detecting selection of anactivation gesture; and in response to detecting selection of theactivation gesture, executing an action associated with a user interfaceelement that corresponds to the current focus.
 3. The method of claim 1,wherein the accessibility information includes one or more wordsoutputted as spoken text that corresponds to the second user interfaceelement.
 4. The method of claim 1, wherein the accessibility informationincludes a border hit sound to indicate that the current focus haschanged from the first user interface element to the second userinterface element.
 5. The method of claim 1, further comprising:detecting a second finger gesture on the touch-sensitive surface,wherein the second finger gesture is independent of contacting alocation on the touch-sensitive surface that corresponds to a third userinterface element, and wherein the second finger gesture issubstantially similar to the first finger gesture; in response todetecting the second finger gesture: changing the current focus from thesecond user interface element in the plurality of user interfaceelements to the third user interface element in the plurality of userinterface elements; and outputting accessibility information associatedwith the third user interface element.
 6. The method of claim 5, whereinthe accessibility information associated with the third user interfaceelement includes a wrapping sound if the third user interface element iseither above or below the second user interface element on the display.7. The method of claim 5, further comprising: detecting a third fingergesture on the touch-sensitive surface, wherein the third finger gestureis independent of contacting the location on the touch-sensitive surfacethat corresponds to the second user interface element, and wherein thethird finger gesture is substantially opposite to the first fingergesture; in response to detecting the third finger gesture: changing thecurrent focus from the third user interface element in the plurality ofuser interface elements to the second user interface element in theplurality of user interface elements; and outputting accessibilityinformation associated with the second user interface element.
 8. Themethod of claim 1, further comprising: if the second user interfaceelement is determined to be a first editable text field: setting acursor at a first cursor location in the first editable text field; andoutputting accessibility information associated with the first editabletext field.
 9. The method of claim 8, further comprising: detecting afourth finger gesture on the touch-sensitive surface in a directionsubstantially perpendicular to the first gesture; in response todetecting the fourth finger gesture: if the cursor is not at the end ofthe first editable text field: moving the cursor forward in the firsteditable text field to a second cursor location within the firsteditable text field; and outputting accessibility information associatedwith the second cursor location within the first editable text field;and, if the cursor is at the end of the first editable text field:maintaining the cursor at the end of the first editable text field; andoutputting accessibility information indicating that the cursor is atthe end of the first editable text field.
 10. An accessible electronicdevice, comprising: a touch-sensitive surface; a display; one or moreprocessors; memory; and one or more programs, wherein the one or moreprograms are stored in the memory and configured to be executed by theone or more processors, the one or more programs including instructionsfor: displaying a plurality of user interface elements on the display,wherein a current focus is on a first user interface element; detectinga first finger gesture directly contacting on the touch-sensitivesurface, wherein the first finger gesture is remote from a location onthe touch-sensitive surface that corresponds to a second user interfaceelement; and, in response to detecting the first finger gesture:changing the current focus from the first user interface element in theplurality of user interface elements to the second user interfaceelement in the plurality of user interface elements; and outputtingaudible accessibility information associated with the second userinterface element.
 11. A non-transitory computer readable storage mediumhaving stored therein instructions, which when executed by an accessibleelectronic device with a touch-sensitive surface and a display, causethe device to: display a plurality of user interface elements on thedisplay, wherein a current focus is on a first user interface element;detect a first finger gesture directly contacting on the touch-sensitivesurface, wherein the first finger gesture is remote from a location onthe touch-sensitive surface that corresponds to a second user interfaceelement; and, in response to detecting the first finger gesture: changethe current focus from the first user interface element in the pluralityof user interface elements to the second user interface element in theplurality of user interface elements; and output audible accessibilityinformation associated with the second user interface element.
 12. Theaccessible electronic device of claim 10, including instructions for:detecting selection of an activation gesture; and in response todetecting selection of the activation gesture, executing an actionassociated with a user interface element that corresponds to the currentfocus.
 13. The accessible electronic device of claim 10, wherein theaccessibility information includes one or more words outputted as spokentext that corresponds to the second user interface element.
 14. Theaccessible electronic device of claim 10, wherein the accessibilityinformation includes a border hit sound to indicate that the currentfocus has changed from the first user interface element to the seconduser interface element.
 15. The accessible electronic device of claim10, including instructions for: detecting a second finger gesture on thetouch-sensitive surface, wherein the second finger gesture isindependent of contacting a location on the touch-sensitive surface thatcorresponds to a third user interface element, and wherein the secondfinger gesture is substantially similar to the first finger gesture; inresponse to detecting the second finger gesture: changing the currentfocus from the second user interface element in the plurality of userinterface elements to the third user interface element in the pluralityof user interface elements; and outputting accessibility informationassociated with the third user interface element.
 16. The accessibleelectronic device of claim 15, wherein the accessibility informationassociated with the third user interface element includes a wrappingsound if the third user interface element is either above or below thesecond user interface element on the display.
 17. The accessibleelectronic device of claim 15, including instructions for: detecting athird finger gesture on the touch-sensitive surface, wherein the thirdfinger gesture is independent of contacting the location on thetouch-sensitive surface that corresponds to the second user interfaceelement, and wherein the third finger gesture is substantially oppositeto the first finger gesture; in response to detecting the third fingergesture: changing the current focus from the third user interfaceelement in the plurality of user interface elements to the second userinterface element in the plurality of user interface elements; andoutputting accessibility information associated with the second userinterface element.
 18. The accessible electronic device of claim 10,including instructions for: if the second user interface element isdetermined to be a first editable text field: setting a cursor at afirst cursor location in the first editable text field; and outputtingaccessibility information associated with the first editable text field.19. The accessible electronic device of claim 18, including instructionsfor: detecting a fourth finger gesture on the touch-sensitive surface ina direction substantially perpendicular to the first gesture; inresponse to detecting the fourth finger gesture: if the cursor is not atthe end of the first editable text field: moving the cursor forward inthe first editable text field to a second cursor location within thefirst editable text field; and outputting accessibility informationassociated with the second cursor location within the first editabletext field; and, if the cursor is at the end of the first editable textfield: maintaining the cursor at the end of the first editable textfield; and outputting accessibility information indicating that thecursor is at the end of the first editable text field.
 20. Thenon-transitory computer readable storage medium of claim 11, includinginstructions which cause the device to: detect selection of anactivation gesture; and in response to detecting selection of theactivation gesture, execute an action associated with a user interfaceelement that corresponds to the current focus.
 21. The non-transitorycomputer readable storage medium of claim 11, wherein the accessibilityinformation includes one or more words outputted as spoken text thatcorresponds to the second user interface element.
 22. The non-transitorycomputer readable storage medium of claim 11, wherein the accessibilityinformation includes a border hit sound to indicate that the currentfocus has changed from the first user interface element to the seconduser interface element.
 23. The non-transitory computer readable storagemedium of claim 11, including instructions which cause the device to:detect a second finger gesture on the touch-sensitive surface, whereinthe second finger gesture is independent of contacting a location on thetouch-sensitive surface that corresponds to a third user interfaceelement, and wherein the second finger gesture is substantially similarto the first finger gesture; in response to detecting the second fingergesture: change the current focus from the second user interface elementin the plurality of user interface elements to the third user interfaceelement in the plurality of user interface elements; and outputaccessibility information associated with the third user interfaceelement.
 24. The non-transitory computer readable storage medium ofclaim 23, wherein the accessibility information associated with thethird user interface element includes a wrapping sound if the third userinterface element is either above or below the second user interfaceelement on the display.
 25. The non-transitory computer readable storagemedium of claim 23, including instructions which cause the device to:detect a third finger gesture on the touch-sensitive surface, whereinthe third finger gesture is independent of contacting the location onthe touch-sensitive surface that corresponds to the second userinterface element, and wherein the third finger gesture is substantiallyopposite to the first finger gesture; in response to detecting the thirdfinger gesture: change the current focus from the third user interfaceelement in the plurality of user interface elements to the second userinterface element in the plurality of user interface elements; andoutput accessibility information associated with the second userinterface element.
 26. The non-transitory computer readable storagemedium of claim 11, including instructions which cause the device to: ifthe second user interface element is determined to be a first editabletext field: set a cursor at a first cursor location in the firsteditable text field; and output accessibility information associatedwith the first editable text field.
 27. The non-transitory computerreadable storage medium of claim 26, including instructions which causethe device to: detect a fourth finger gesture on the touch-sensitivesurface in a direction substantially perpendicular to the first gesture;in response to detecting the fourth finger gesture: if the cursor is notat the end of the first editable text field: move the cursor forward inthe first editable text field to a second cursor location within thefirst editable text field; and output accessibility informationassociated with the second cursor location within the first editabletext field; and, if the cursor is at the end of the first editable textfield: maintain the cursor at the end of the first editable text field;and output accessibility information indicating that the cursor is atthe end of the first editable text field.